[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:25:39 dist-test-slave-fd7l krb5kdc[2537](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:25:39 dist-test-slave-fd7l krb5kdc[2537](info): set up 2 sockets
Aug 11 00:25:39 dist-test-slave-fd7l krb5kdc[2537](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 00:25:40.561300 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.003s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:25:40 dist-test-slave-fd7l krb5kdc[2537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:25:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:25:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:25:40.778692 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:43549
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:43549
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:41.169857 2554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:41.170207 2554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:41.170353 2554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:41.178308 2554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:25:41.178464 2554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:41.178520 2554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:41.178579 2554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:25:41.178663 2554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:25:41.190606 2554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:43549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:43549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:41.192287 2554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:41.193928 2554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:41.208480 2559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:41.208421 2562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:41.208735 2560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:41.209308 2554 server_base.cc:1034] running on GCE node
I20260811 00:25:41.210181 2554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:41.212131 2554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:41.213377 2554 hybrid_clock.cc:648] HybridClock initialized: now 1786407941213296 us; error 77 us; skew 500 ppm
I20260811 00:25:41.213755 2554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:41 dist-test-slave-fd7l krb5kdc[2537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407941, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:41.218261 2554 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:25:41.220108 2554 webserver.cc:460] Webserver started at http://127.2.120.62:46605/ using document root <none> and password file <none>
I20260811 00:25:41.221056 2554 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:41.221223 2554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:41.221581 2554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:41.225536 2554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "f48ae87f10634feea13dbe06a7489841"
format_stamp: "Formatted at 2026-08-11 00:25:41 on dist-test-slave-fd7l"
server_key: "2020bfcccb62b5058aa8af5a70c51211"
server_key_iv: "5b2404395d68727977e7573a0fea18e6"
server_key_version: "encryptionkey@0"
I20260811 00:25:41.226441 2554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "f48ae87f10634feea13dbe06a7489841"
format_stamp: "Formatted at 2026-08-11 00:25:41 on dist-test-slave-fd7l"
server_key: "2020bfcccb62b5058aa8af5a70c51211"
server_key_iv: "5b2404395d68727977e7573a0fea18e6"
server_key_version: "encryptionkey@0"
I20260811 00:25:41.232820 2554 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 00:25:41.236891 2571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:41.238399 2554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:25:41.238626 2554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "f48ae87f10634feea13dbe06a7489841"
format_stamp: "Formatted at 2026-08-11 00:25:41 on dist-test-slave-fd7l"
server_key: "2020bfcccb62b5058aa8af5a70c51211"
server_key_iv: "5b2404395d68727977e7573a0fea18e6"
server_key_version: "encryptionkey@0"
I20260811 00:25:41.238901 2554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:41.260849 2554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:41.265997 2554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:41.266316 2554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:41.288837 2554 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:43549
I20260811 00:25:41.288900 2627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:43549 every 8 connection(s)
I20260811 00:25:41.290400 2554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:25:41.292094 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 2554
I20260811 00:25:41.292487 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:25:41.294493 2528 external_mini_cluster.cc:1428] Setting key 0a0a95e6e1489f2fa08285705aef383b
I20260811 00:25:41.295226 2628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:41.308037 2628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841: Bootstrap starting.
I20260811 00:25:41.311590 2628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:41.312731 2628 log.cc:824] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:41.315725 2628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841: No bootstrap required, opened a new log
Aug 11 00:25:41 dist-test-slave-fd7l krb5kdc[2537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:25:41.324144 2628 raft_consensus.cc:348] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f48ae87f10634feea13dbe06a7489841" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43549 } }
I20260811 00:25:41.324604 2628 raft_consensus.cc:374] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:41.324708 2628 raft_consensus.cc:729] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f48ae87f10634feea13dbe06a7489841, State: Initialized, Role: FOLLOWER
I20260811 00:25:41.325413 2628 consensus_queue.cc:260] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f48ae87f10634feea13dbe06a7489841" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43549 } }
I20260811 00:25:41.325765 2628 raft_consensus.cc:388] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:25:41.326000 2628 raft_consensus.cc:482] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:25:41.326213 2628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:41.329140 2628 raft_consensus.cc:504] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f48ae87f10634feea13dbe06a7489841" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43549 } }
I20260811 00:25:41.329909 2628 leader_election.cc:302] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f48ae87f10634feea13dbe06a7489841; no voters:
I20260811 00:25:41.330832 2628 leader_election.cc:288] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:25:41.331077 2633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:41.332902 2633 raft_consensus.cc:686] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841 [term 1 LEADER]: Becoming Leader. State: Replica: f48ae87f10634feea13dbe06a7489841, State: Running, Role: LEADER
I20260811 00:25:41.333595 2633 consensus_queue.cc:237] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f48ae87f10634feea13dbe06a7489841" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43549 } }
I20260811 00:25:41.334115 2628 sys_catalog.cc:564] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:25:41.337394 2630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:41.298758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32914 (local address 127.2.120.62:43549)
0811 00:25:41.299680 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:41.299692 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:41.301320 (+ 1628us) server_negotiation.cc:407] Connection header received
0811 00:25:41.310375 (+ 9055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:41.310394 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:41.310751 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:41.311188 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:41.314726 (+ 3538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:41.316052 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:41.317770 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:41.318457 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:41.318700 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:41.325227 (+ 6527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:41.325245 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:41.325272 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:41.325330 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:41.329625 (+ 4295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:41.330800 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:41.330810 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:41.330816 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:41.331271 (+ 455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:41.332131 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:41.332137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:41.332140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:41.333223 (+ 1083us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:41.333593 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:41.335037 (+ 1444us) server_negotiation.cc:299] Negotiation successful
0811 00:25:41.335523 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":430,"threads_started":1}
I20260811 00:25:41.343210 2635 sys_catalog.cc:455] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f48ae87f10634feea13dbe06a7489841" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f48ae87f10634feea13dbe06a7489841" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43549 } } }
I20260811 00:25:41.344118 2635 sys_catalog.cc:458] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:41.343428 2636 sys_catalog.cc:455] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f48ae87f10634feea13dbe06a7489841. Latest consensus state: current_term: 1 leader_uuid: "f48ae87f10634feea13dbe06a7489841" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f48ae87f10634feea13dbe06a7489841" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43549 } } }
I20260811 00:25:41.345032 2636 sys_catalog.cc:458] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:41.345860 2642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:25:41.356484 2642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:25:41.366432 2642 catalog_manager.cc:1347] Generated new cluster ID: 3c73c0d1ed0147af97a12717b30c18eb
I20260811 00:25:41.366572 2642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:25:41.375164 2642 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:25:41.377151 2642 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:25:41.384799 2642 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f48ae87f10634feea13dbe06a7489841: Generated new TSK 0
I20260811 00:25:41.386034 2642 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
I20260811 00:25:41.444933 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43549
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:41.886518 2657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:41.886940 2657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:41.887142 2657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:41.900128 2657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:41.900401 2657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:41.900709 2657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:25:41.913803 2657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43549
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:41.915227 2657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:41.916659 2657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:41.930833 2663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:41.931169 2664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:41.931999 2666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:41.933374 2657 server_base.cc:1034] running on GCE node
I20260811 00:25:41.934041 2657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:41.935122 2657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:41.936342 2657 hybrid_clock.cc:648] HybridClock initialized: now 1786407941936269 us; error 58 us; skew 500 ppm
I20260811 00:25:41.936707 2657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:41 dist-test-slave-fd7l krb5kdc[2537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407941, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:41.940945 2657 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:25:41.943198 2657 webserver.cc:460] Webserver started at http://127.2.120.1:32817/ using document root <none> and password file <none>
I20260811 00:25:41.943991 2657 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:41.944103 2657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:41.944383 2657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:41.947561 2657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "31a353fa80844f079407336e57b3ff18"
format_stamp: "Formatted at 2026-08-11 00:25:41 on dist-test-slave-fd7l"
server_key: "c184d4172d051a252dca20bac21f49a0"
server_key_iv: "ab686807d87d5e4fc5905ab15676dddc"
server_key_version: "encryptionkey@0"
I20260811 00:25:41.948592 2657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "31a353fa80844f079407336e57b3ff18"
format_stamp: "Formatted at 2026-08-11 00:25:41 on dist-test-slave-fd7l"
server_key: "c184d4172d051a252dca20bac21f49a0"
server_key_iv: "ab686807d87d5e4fc5905ab15676dddc"
server_key_version: "encryptionkey@0"
I20260811 00:25:41.954135 2657 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 00:25:41.958307 2674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:41.959896 2657 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:25:41.960071 2657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "31a353fa80844f079407336e57b3ff18"
format_stamp: "Formatted at 2026-08-11 00:25:41 on dist-test-slave-fd7l"
server_key: "c184d4172d051a252dca20bac21f49a0"
server_key_iv: "ab686807d87d5e4fc5905ab15676dddc"
server_key_version: "encryptionkey@0"
I20260811 00:25:41.960282 2657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:41.977551 2657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:41.982630 2657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:41.982920 2657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:41.984145 2657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:25:41.985883 2657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:25:41.985970 2657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:41.986073 2657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:25:41.986119 2657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:42.034162 2657 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:41545
I20260811 00:25:42.034198 2803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:41545 every 8 connection(s)
I20260811 00:25:42.035712 2657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:25:42.040872 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 2657
I20260811 00:25:42.041137 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:25:42.041970 2528 external_mini_cluster.cc:1428] Setting key ebaefe3d072f300f07e00a90e835638a
Aug 11 00:25:42 dist-test-slave-fd7l krb5kdc[2537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407941, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.120.2@KRBTEST.COM; defaulting to no policy
I20260811 00:25:42.061476 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:42.038651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:35703 (local address 127.2.120.62:43549)
0811 00:25:42.039179 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:42.039184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:42.040158 (+ 974us) server_negotiation.cc:407] Connection header received
0811 00:25:42.043357 (+ 3199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:42.043364 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:42.043460 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:42.043595 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:42.045738 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:42.046587 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:42.049352 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:42.049793 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:42.049958 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:42.054109 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:42.054114 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:42.054118 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:42.054154 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:42.057558 (+ 3404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:42.058625 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:42.058630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:42.058632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:42.058777 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:42.059410 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:42.059414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:42.059416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:42.059741 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:42.059906 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:42.060796 (+ 890us) server_negotiation.cc:299] Negotiation successful
0811 00:25:42.061048 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":296,"threads_started":1}
I20260811 00:25:42.062739 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:42.039029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:43549 (local address 127.2.120.1:35703)
0811 00:25:42.039877 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:42.039919 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:42.042967 (+ 3048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:42.043977 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:42.043994 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:42.044846 (+ 852us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:42.045535 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:42.045557 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:42.046846 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:42.046855 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:42.049168 (+ 2313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:42.049184 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:42.050180 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:42.050190 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:42.050570 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:42.051418 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:42.051444 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:42.053820 (+ 2376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:42.057748 (+ 3928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:42.057759 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:42.057787 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:42.058475 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:42.058896 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:42.058901 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:42.058905 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:42.059274 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:42.059879 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:42.059888 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:42.060333 (+ 445us) client_negotiation.cc:769] Sending connection context
0811 00:25:42.060708 (+ 375us) client_negotiation.cc:240] Negotiation successful
0811 00:25:42.061389 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":351,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.2@KRBTEST.COM" created.
I20260811 00:25:42.064649 2804 heartbeater.cc:347] Connected to a master server at 127.2.120.62:43549
I20260811 00:25:42.064994 2804 heartbeater.cc:464] Registering TS with master...
I20260811 00:25:42.065882 2804 heartbeater.cc:511] Master 127.2.120.62:43549 requested a full tablet report, sending...
I20260811 00:25:42.068594 2588 ts_manager.cc:194] Registered new tserver with Master: 31a353fa80844f079407336e57b3ff18 (127.2.120.1:41545)
I20260811 00:25:42.071097 2588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:35703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
I20260811 00:25:42.092478 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43549
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:42.483028 2810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:42.483350 2810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:42.483531 2810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:42.491513 2810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:42.491654 2810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:42.491851 2810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:25:42.508307 2810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43549
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:42.511605 2810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:42.516602 2810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:42.534091 2815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:42.534029 2816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:42.534155 2818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:42.535557 2810 server_base.cc:1034] running on GCE node
I20260811 00:25:42.536355 2810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:42.537577 2810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:42.538868 2810 hybrid_clock.cc:648] HybridClock initialized: now 1786407942538786 us; error 72 us; skew 500 ppm
I20260811 00:25:42.539290 2810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:42 dist-test-slave-fd7l krb5kdc[2537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407942, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:42.543183 2810 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:25:42.545373 2810 webserver.cc:460] Webserver started at http://127.2.120.2:39567/ using document root <none> and password file <none>
I20260811 00:25:42.546839 2810 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:42.547066 2810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:42.547514 2810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:42.551084 2810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "4a9c354fc647471eb1449d9af961568d"
format_stamp: "Formatted at 2026-08-11 00:25:42 on dist-test-slave-fd7l"
server_key: "92ea7c56e7ee3959b390c5ef8cdb0592"
server_key_iv: "dfc0b0a696627b8fc2b758b1b563a4db"
server_key_version: "encryptionkey@0"
I20260811 00:25:42.552052 2810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "4a9c354fc647471eb1449d9af961568d"
format_stamp: "Formatted at 2026-08-11 00:25:42 on dist-test-slave-fd7l"
server_key: "92ea7c56e7ee3959b390c5ef8cdb0592"
server_key_iv: "dfc0b0a696627b8fc2b758b1b563a4db"
server_key_version: "encryptionkey@0"
I20260811 00:25:42.559392 2810 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260811 00:25:42.563678 2826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:42.565490 2810 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 00:25:42.565760 2810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "4a9c354fc647471eb1449d9af961568d"
format_stamp: "Formatted at 2026-08-11 00:25:42 on dist-test-slave-fd7l"
server_key: "92ea7c56e7ee3959b390c5ef8cdb0592"
server_key_iv: "dfc0b0a696627b8fc2b758b1b563a4db"
server_key_version: "encryptionkey@0"
I20260811 00:25:42.566045 2810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:42.578962 2810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:42.584143 2810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:42.584496 2810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:42.585618 2810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:25:42.587078 2810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:25:42.587188 2810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:42.587335 2810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:25:42.587405 2810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:42.627254 2810 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:33367
I20260811 00:25:42.627305 2957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:33367 every 8 connection(s)
I20260811 00:25:42.628652 2810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:25:42.636117 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 2810
I20260811 00:25:42.636329 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:25:42.636852 2528 external_mini_cluster.cc:1428] Setting key b8c0567ccdc4137399baefc5a6f12fb8
Aug 11 00:25:42 dist-test-slave-fd7l krb5kdc[2537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407942, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:25:42.652329 2961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:42.631510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:34995 (local address 127.2.120.62:43549)
0811 00:25:42.632072 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:42.632081 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:42.633277 (+ 1196us) server_negotiation.cc:407] Connection header received
0811 00:25:42.635378 (+ 2101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:42.635383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:42.635492 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:42.635636 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:42.637960 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:42.638812 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:42.640631 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:42.641111 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:42.641264 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:42.645313 (+ 4049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:42.645318 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:42.645321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:42.645356 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:42.648661 (+ 3305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:42.649756 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:42.649761 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:42.649763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:42.649904 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:42.650453 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:42.650457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:42.650459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:42.650768 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:42.650895 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:42.651890 (+ 995us) server_negotiation.cc:299] Negotiation successful
0811 00:25:42.652077 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":312,"threads_started":1}
I20260811 00:25:42.654021 2962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:42.632149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:43549 (local address 127.2.120.2:34995)
0811 00:25:42.632962 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:42.633002 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:42.635028 (+ 2026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:42.635950 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:42.635960 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:42.636852 (+ 892us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:42.637733 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:42.637774 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:42.638997 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:42.639009 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:42.640398 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:42.640418 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:42.641372 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:42.641380 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:42.641643 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:42.642514 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:42.642545 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:42.645059 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:42.648796 (+ 3737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:42.648809 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:42.648830 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:42.649578 (+ 748us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:42.650042 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:42.650050 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:42.650056 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:42.650330 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:42.650895 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:42.650907 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:42.651448 (+ 541us) client_negotiation.cc:769] Sending connection context
0811 00:25:42.651810 (+ 362us) client_negotiation.cc:240] Negotiation successful
0811 00:25:42.652417 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":367,"threads_started":1}
I20260811 00:25:42.655900 2958 heartbeater.cc:347] Connected to a master server at 127.2.120.62:43549
I20260811 00:25:42.656263 2958 heartbeater.cc:464] Registering TS with master...
I20260811 00:25:42.657400 2958 heartbeater.cc:511] Master 127.2.120.62:43549 requested a full tablet report, sending...
I20260811 00:25:42.659112 2588 ts_manager.cc:194] Registered new tserver with Master: 4a9c354fc647471eb1449d9af961568d (127.2.120.2:33367)
WARNING: no policy specified for kudu/127.2.120.3@KRBTEST.COM; defaulting to no policy
I20260811 00:25:42.660203 2588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:34995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
I20260811 00:25:42.688827 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43549
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:43.066918 2965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:43.067245 2965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:43.067414 2965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 00:25:43.074350 2804 heartbeater.cc:503] Master 127.2.120.62:43549 was elected leader, sending a full tablet report...
W20260811 00:25:43.075660 2965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:43.075842 2965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:43.076030 2965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:25:43.086750 2965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43549
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:43.088315 2965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:43.089777 2965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:43.104650 2973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:43.104580 2976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:43.104571 2972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:43.105830 2965 server_base.cc:1034] running on GCE node
I20260811 00:25:43.106596 2965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:43.107636 2965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:43.108862 2965 hybrid_clock.cc:648] HybridClock initialized: now 1786407943108817 us; error 41 us; skew 500 ppm
I20260811 00:25:43.109205 2965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:43 dist-test-slave-fd7l krb5kdc[2537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407943, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:43.112578 2965 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:25:43.114389 2965 webserver.cc:460] Webserver started at http://127.2.120.3:33791/ using document root <none> and password file <none>
I20260811 00:25:43.115113 2965 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:43.115262 2965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:43.115602 2965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:43.118839 2965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "8329781743114b779cd85e6d9f48e021"
format_stamp: "Formatted at 2026-08-11 00:25:43 on dist-test-slave-fd7l"
server_key: "4dba40701640b0c56dbbaa66f7d8296a"
server_key_iv: "42e9d6090ec759b2b47588935ca873c5"
server_key_version: "encryptionkey@0"
I20260811 00:25:43.119663 2965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "8329781743114b779cd85e6d9f48e021"
format_stamp: "Formatted at 2026-08-11 00:25:43 on dist-test-slave-fd7l"
server_key: "4dba40701640b0c56dbbaa66f7d8296a"
server_key_iv: "42e9d6090ec759b2b47588935ca873c5"
server_key_version: "encryptionkey@0"
I20260811 00:25:43.125385 2965 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 00:25:43.129642 2983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:43.131362 2965 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:25:43.131603 2965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "8329781743114b779cd85e6d9f48e021"
format_stamp: "Formatted at 2026-08-11 00:25:43 on dist-test-slave-fd7l"
server_key: "4dba40701640b0c56dbbaa66f7d8296a"
server_key_iv: "42e9d6090ec759b2b47588935ca873c5"
server_key_version: "encryptionkey@0"
I20260811 00:25:43.131853 2965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:43.148911 2965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:43.154451 2965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:43.154752 2965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:43.156337 2965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:25:43.158056 2965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:25:43.158150 2965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:43.158255 2965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:25:43.158298 2965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:43.206728 2965 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:38467
I20260811 00:25:43.206741 3113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:38467 every 8 connection(s)
I20260811 00:25:43.208443 2965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:25:43.210402 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 2965
I20260811 00:25:43.210624 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:25:43.211077 2528 external_mini_cluster.cc:1428] Setting key 67906a5a3c6a9aef4791804cddf20340
Aug 11 00:25:43 dist-test-slave-fd7l krb5kdc[2537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407943, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:25:43.236094 3116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:43.212340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:43531 (local address 127.2.120.62:43549)
0811 00:25:43.213084 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:43.213090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:43.214116 (+ 1026us) server_negotiation.cc:407] Connection header received
0811 00:25:43.216666 (+ 2550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:43.216672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:43.216774 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:43.216897 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:43.218741 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:43.219559 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:43.221632 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:43.222140 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:43.222300 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:43.227816 (+ 5516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:43.227822 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:43.227826 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:43.227869 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:43.231715 (+ 3846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:43.232860 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:43.232866 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:43.232869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:43.233048 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:43.233631 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:43.233636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:43.233638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:43.234082 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:43.234283 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:43.235147 (+ 864us) server_negotiation.cc:299] Negotiation successful
0811 00:25:43.235468 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":375,"threads_started":1}
I20260811 00:25:43.237314 3117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:43.212847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:43549 (local address 127.2.120.3:43531)
0811 00:25:43.213832 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:43.213881 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:43.216296 (+ 2415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:43.217183 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:43.217193 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:43.217830 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:43.218541 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:43.218565 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:43.219739 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:43.219743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:43.221431 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:43.221448 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:43.222429 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:43.222438 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:43.222810 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:43.223729 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:43.223767 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:43.227542 (+ 3775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:43.231968 (+ 4426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:43.231980 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:43.232000 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:43.232686 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:43.233197 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:43.233203 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:43.233207 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:43.233473 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:43.234239 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:43.234250 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:43.234697 (+ 447us) client_negotiation.cc:769] Sending connection context
0811 00:25:43.235062 (+ 365us) client_negotiation.cc:240] Negotiation successful
0811 00:25:43.235792 (+ 730us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":389,"threads_started":1}
I20260811 00:25:43.239298 3114 heartbeater.cc:347] Connected to a master server at 127.2.120.62:43549
I20260811 00:25:43.239663 3114 heartbeater.cc:464] Registering TS with master...
I20260811 00:25:43.240556 3114 heartbeater.cc:511] Master 127.2.120.62:43549 requested a full tablet report, sending...
I20260811 00:25:43.242740 2588 ts_manager.cc:194] Registered new tserver with Master: 8329781743114b779cd85e6d9f48e021 (127.2.120.3:38467)
I20260811 00:25:43.243966 2588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:43531
I20260811 00:25:43.258587 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 00:25:43 dist-test-slave-fd7l krb5kdc[2537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407943, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 00:25:43 dist-test-slave-fd7l krb5kdc[2537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407943, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:25:43.323253 3123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:43.305595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48490 (local address 127.2.120.1:41545)
0811 00:25:43.306631 (+ 1036us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:43.306641 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:43.306678 (+ 37us) server_negotiation.cc:407] Connection header received
0811 00:25:43.306865 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:43.306874 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:43.307111 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:43.307286 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:43.309170 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:43.310211 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:43.311587 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:43.312195 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:43.312348 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:43.317029 (+ 4681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:43.317043 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:43.317059 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:43.317104 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:43.320177 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:43.321005 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:43.321013 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:43.321016 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:43.321154 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:43.321571 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:43.321576 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:43.321578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:43.321948 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:43.322289 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:43.322596 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 00:25:43.322886 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":471,"threads_started":1}
W20260811 00:25:43.324822 2708 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48490
Aug 11 00:25:43 dist-test-slave-fd7l krb5kdc[2537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 00:25:43 dist-test-slave-fd7l krb5kdc[2537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:25:43.380057 3123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:43.365505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41964 (local address 127.2.120.1:41545)
0811 00:25:43.365735 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:43.365744 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:43.365912 (+ 168us) server_negotiation.cc:407] Connection header received
0811 00:25:43.366046 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:43.366050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:43.366120 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:43.366215 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:43.367738 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:43.368468 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:43.369819 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:43.370263 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:43.370374 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:43.374634 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:43.374639 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:43.374643 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:43.374687 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:43.377592 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:43.378282 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:43.378287 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:43.378290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:43.378435 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:43.378764 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:43.378767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:43.378770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:43.379020 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:43.379227 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:43.379519 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 00:25:43.379703 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 00:25:43.382088 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 2657
I20260811 00:25:43.396028 3123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:43.383071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41972 (local address 127.2.120.1:41545)
0811 00:25:43.383239 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:43.383243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:43.383567 (+ 324us) server_negotiation.cc:407] Connection header received
0811 00:25:43.383698 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:43.383702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:43.383763 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:43.383849 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:43.385350 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:43.386193 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:43.387501 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:43.387952 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:43.388068 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:43.390519 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:43.390524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:43.390528 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:43.390564 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:43.393406 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:43.394103 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:43.394108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:43.394110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:43.394253 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:43.394681 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:43.394686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:43.394688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:43.395021 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:43.395240 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:43.395519 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 00:25:43.395711 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 00:25:43.396941 2798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:25:43.489189 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 2657
I20260811 00:25:43.512885 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 2810
Aug 11 00:25:43 dist-test-slave-fd7l krb5kdc[2537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.2@KRBTEST.COM
I20260811 00:25:43.531752 3130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:43.513856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35246 (local address 127.2.120.2:33367)
0811 00:25:43.514407 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:43.514414 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:43.514447 (+ 33us) server_negotiation.cc:407] Connection header received
0811 00:25:43.514512 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:43.514519 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:43.514737 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:43.514922 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:43.516895 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:43.517891 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:43.519210 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:43.519785 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:43.519947 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:43.524690 (+ 4743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:43.524712 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:43.524736 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:43.524804 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:43.528188 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:43.529337 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:43.529346 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:43.529349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:43.529543 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:43.529990 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:43.529994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:43.529996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:43.530393 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:43.530664 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:43.531044 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 00:25:43.531377 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":274,"threads_started":1}
I20260811 00:25:43.533313 2953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:25:43.627408 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 2810
I20260811 00:25:43.650775 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 2965
Aug 11 00:25:43 dist-test-slave-fd7l krb5kdc[2537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.3@KRBTEST.COM
I20260811 00:25:43.668361 3132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:43.651883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48472 (local address 127.2.120.3:38467)
0811 00:25:43.652392 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:43.652399 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:43.652429 (+ 30us) server_negotiation.cc:407] Connection header received
0811 00:25:43.652499 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:43.652544 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:43.652779 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:43.652955 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:43.654238 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:43.655400 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:43.656982 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:43.657586 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:43.657748 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:43.662219 (+ 4471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:43.662232 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:43.662250 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:43.662292 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:43.665622 (+ 3330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:43.666356 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:43.666364 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:43.666366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:43.666514 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:43.666885 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:43.666888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:43.666890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:43.667276 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:43.667529 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:43.667838 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 00:25:43.668048 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":249,"threads_started":1}
I20260811 00:25:43.669795 3109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:25:43.755379 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 2965
I20260811 00:25:43.778160 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 2554
I20260811 00:25:43.779526 2621 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:25:43.886343 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 2554
2026-08-11T00:25:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4585 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:25:43 dist-test-slave-fd7l krb5kdc[3137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:25:43 dist-test-slave-fd7l krb5kdc[3137](info): set up 2 sockets
Aug 11 00:25:43 dist-test-slave-fd7l krb5kdc[3137](info): commencing operation
krb5kdc: starting...
W20260811 00:25:45.090764 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:25:45 dist-test-slave-fd7l krb5kdc[3137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:25:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:25:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:25:45.306475 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:35005
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:35005
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:45.676477 3151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:45.676905 3151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:45.677003 3151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:45.684716 3151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:25:45.684846 3151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:45.684893 3151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:45.684944 3151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:25:45.684983 3151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:25:45.695436 3151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:35005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:35005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:45.696856 3151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:45.698266 3151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:45.711223 3157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:45.711258 3158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:45.711313 3161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:45.712345 3151 server_base.cc:1034] running on GCE node
I20260811 00:25:45.713253 3151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:45.714793 3151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:45.716017 3151 hybrid_clock.cc:648] HybridClock initialized: now 1786407945715969 us; error 35 us; skew 500 ppm
I20260811 00:25:45.716365 3151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:45 dist-test-slave-fd7l krb5kdc[3137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407945, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:45.728317 3151 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:25:45.730271 3151 webserver.cc:460] Webserver started at http://127.2.120.62:37777/ using document root <none> and password file <none>
I20260811 00:25:45.731073 3151 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:45.731211 3151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:45.731573 3151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:45.734925 3151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "9ba4f9a53d3a4afc9a121b87502d91bb"
format_stamp: "Formatted at 2026-08-11 00:25:45 on dist-test-slave-fd7l"
server_key: "f6e24c0142e06905fa547220320a9abe"
server_key_iv: "599daa81728a6dc00535262a6ccae146"
server_key_version: "encryptionkey@0"
I20260811 00:25:45.735903 3151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "9ba4f9a53d3a4afc9a121b87502d91bb"
format_stamp: "Formatted at 2026-08-11 00:25:45 on dist-test-slave-fd7l"
server_key: "f6e24c0142e06905fa547220320a9abe"
server_key_iv: "599daa81728a6dc00535262a6ccae146"
server_key_version: "encryptionkey@0"
I20260811 00:25:45.741860 3151 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 00:25:45.745699 3169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:45.747210 3151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:25:45.747407 3151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "9ba4f9a53d3a4afc9a121b87502d91bb"
format_stamp: "Formatted at 2026-08-11 00:25:45 on dist-test-slave-fd7l"
server_key: "f6e24c0142e06905fa547220320a9abe"
server_key_iv: "599daa81728a6dc00535262a6ccae146"
server_key_version: "encryptionkey@0"
I20260811 00:25:45.747632 3151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:45.766567 3151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:45.771472 3151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:45.771816 3151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:45.793524 3151 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:35005
I20260811 00:25:45.793561 3221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:35005 every 8 connection(s)
I20260811 00:25:45.795080 3151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:25:45.799242 3223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:45.800123 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 3151
I20260811 00:25:45.800308 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:25:45.801055 2528 external_mini_cluster.cc:1428] Setting key dcc8662b68ca432fd07e580a1820b094
I20260811 00:25:45.816421 3223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb: Bootstrap starting.
Aug 11 00:25:45 dist-test-slave-fd7l krb5kdc[3137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:25:45.821825 3223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:45.823709 3223 log.cc:824] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:45.828253 3223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb: No bootstrap required, opened a new log
I20260811 00:25:45.829777 3226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:45.803524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39112 (local address 127.2.120.62:35005)
0811 00:25:45.804594 (+ 1070us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:45.804609 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:45.804675 (+ 66us) server_negotiation.cc:407] Connection header received
0811 00:25:45.807131 (+ 2456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:45.807178 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:45.807998 (+ 820us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:45.808743 (+ 745us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:45.810878 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:45.812462 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:45.814640 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:45.815552 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:45.815823 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:45.820751 (+ 4928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:45.820761 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:45.820782 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:45.820844 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:45.825125 (+ 4281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:45.825935 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:45.825944 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:45.825952 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:45.826149 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:45.826555 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:45.826565 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:45.826567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:45.827254 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:45.827572 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:45.828105 (+ 533us) server_negotiation.cc:299] Negotiation successful
0811 00:25:45.828668 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":428,"threads_started":1}
I20260811 00:25:45.837724 3223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba4f9a53d3a4afc9a121b87502d91bb" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 35005 } }
I20260811 00:25:45.838234 3223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:45.838354 3223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ba4f9a53d3a4afc9a121b87502d91bb, State: Initialized, Role: FOLLOWER
I20260811 00:25:45.839035 3223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba4f9a53d3a4afc9a121b87502d91bb" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 35005 } }
I20260811 00:25:45.839304 3223 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:25:45.839454 3223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:25:45.839638 3223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:45.842042 3223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba4f9a53d3a4afc9a121b87502d91bb" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 35005 } }
I20260811 00:25:45.842629 3223 leader_election.cc:302] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ba4f9a53d3a4afc9a121b87502d91bb; no voters:
I20260811 00:25:45.843698 3223 leader_election.cc:288] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:25:45.843839 3228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:45.845749 3228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb [term 1 LEADER]: Becoming Leader. State: Replica: 9ba4f9a53d3a4afc9a121b87502d91bb, State: Running, Role: LEADER
I20260811 00:25:45.846549 3228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba4f9a53d3a4afc9a121b87502d91bb" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 35005 } }
I20260811 00:25:45.847079 3223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:25:45.849663 3229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ba4f9a53d3a4afc9a121b87502d91bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba4f9a53d3a4afc9a121b87502d91bb" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 35005 } } }
I20260811 00:25:45.849663 3231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ba4f9a53d3a4afc9a121b87502d91bb. Latest consensus state: current_term: 1 leader_uuid: "9ba4f9a53d3a4afc9a121b87502d91bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba4f9a53d3a4afc9a121b87502d91bb" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 35005 } } }
I20260811 00:25:45.850232 3231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:45.851236 3229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:45.853874 3237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:25:45.860234 3237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:25:45.872928 3237 catalog_manager.cc:1347] Generated new cluster ID: a11fb788557045079e23602be4caac60
I20260811 00:25:45.873087 3237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:25:45.888567 3237 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:25:45.890355 3237 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:25:45.904690 3237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ba4f9a53d3a4afc9a121b87502d91bb: Generated new TSK 0
I20260811 00:25:45.905661 3237 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
I20260811 00:25:45.973914 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:35005
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:46.386439 3256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:46.386793 3256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:46.386969 3256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:46.394594 3256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:46.394763 3256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:46.394948 3256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:25:46.405426 3256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:35005
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:46.406932 3256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:46.408458 3256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:46.422857 3264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:46.422933 3266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:46.422858 3263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:46.423857 3256 server_base.cc:1034] running on GCE node
I20260811 00:25:46.424566 3256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:46.425640 3256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:46.426903 3256 hybrid_clock.cc:648] HybridClock initialized: now 1786407946426822 us; error 74 us; skew 500 ppm
I20260811 00:25:46.427302 3256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:46 dist-test-slave-fd7l krb5kdc[3137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407946, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:46.430739 3256 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:25:46.432296 3256 webserver.cc:460] Webserver started at http://127.2.120.1:46209/ using document root <none> and password file <none>
I20260811 00:25:46.433068 3256 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:46.433213 3256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:46.433558 3256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:46.436697 3256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "cbf74c56200e457c91b84db353142c06"
format_stamp: "Formatted at 2026-08-11 00:25:46 on dist-test-slave-fd7l"
server_key: "6ab02d3e517fffd639fdab6d03031ca9"
server_key_iv: "f38c8afe5bc893eea67536f49ce27702"
server_key_version: "encryptionkey@0"
I20260811 00:25:46.437516 3256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "cbf74c56200e457c91b84db353142c06"
format_stamp: "Formatted at 2026-08-11 00:25:46 on dist-test-slave-fd7l"
server_key: "6ab02d3e517fffd639fdab6d03031ca9"
server_key_iv: "f38c8afe5bc893eea67536f49ce27702"
server_key_version: "encryptionkey@0"
I20260811 00:25:46.442981 3256 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:25:46.446877 3273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:46.448652 3256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:25:46.448855 3256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "cbf74c56200e457c91b84db353142c06"
format_stamp: "Formatted at 2026-08-11 00:25:46 on dist-test-slave-fd7l"
server_key: "6ab02d3e517fffd639fdab6d03031ca9"
server_key_iv: "f38c8afe5bc893eea67536f49ce27702"
server_key_version: "encryptionkey@0"
I20260811 00:25:46.449095 3256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:46.494375 3256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:46.499641 3256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:46.499985 3256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:46.501152 3256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:25:46.502655 3256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:25:46.502759 3256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:46.502915 3256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:25:46.502988 3256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:46.543478 3256 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:43717
I20260811 00:25:46.543509 3405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:43717 every 8 connection(s)
I20260811 00:25:46.547097 3256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:25:46.552314 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 3256
I20260811 00:25:46.552791 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:25:46.553987 2528 external_mini_cluster.cc:1428] Setting key 409a07147b55d5fc13d7814729293683
Aug 11 00:25:46 dist-test-slave-fd7l krb5kdc[3137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407946, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.120.2@KRBTEST.COM; defaulting to no policy
I20260811 00:25:46.587421 3408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:46.554406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:46385 (local address 127.2.120.62:35005)
0811 00:25:46.561662 (+ 7256us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:46.561676 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:46.561748 (+ 72us) server_negotiation.cc:407] Connection header received
0811 00:25:46.564568 (+ 2820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:46.564584 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:46.564836 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:46.565091 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:46.570124 (+ 5033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:46.572457 (+ 2333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:46.575588 (+ 3131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:46.576092 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:46.576262 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:46.580799 (+ 4537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:46.580805 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:46.580809 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:46.580848 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:46.583705 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:46.584660 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:46.584665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:46.584667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:46.584817 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:46.585322 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:46.585326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:46.585328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:46.585721 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:46.585884 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:46.586758 (+ 874us) server_negotiation.cc:299] Negotiation successful
0811 00:25:46.587026 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6927,"thread_start_us":6659,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.2@KRBTEST.COM" created.
I20260811 00:25:46.589075 3411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:46.556136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:35005 (local address 127.2.120.1:46385)
0811 00:25:46.558635 (+ 2499us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:46.558774 (+ 139us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:46.563174 (+ 4400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:46.566004 (+ 2830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:46.566029 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:46.567484 (+ 1455us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:46.569410 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:46.569509 (+ 99us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:46.573135 (+ 3626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:46.573146 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:46.575276 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:46.575311 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:46.576411 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:46.576419 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:46.576774 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:46.577559 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:46.577590 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:46.580374 (+ 2784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:46.583909 (+ 3535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:46.583922 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:46.583951 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:46.584491 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:46.584941 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:46.584947 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:46.584951 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:46.585184 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:46.585861 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:46.585870 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:46.586289 (+ 419us) client_negotiation.cc:769] Sending connection context
0811 00:25:46.586614 (+ 325us) client_negotiation.cc:240] Negotiation successful
0811 00:25:46.587249 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1534,"thread_start_us":878,"threads_started":1}
I20260811 00:25:46.591064 3406 heartbeater.cc:347] Connected to a master server at 127.2.120.62:35005
I20260811 00:25:46.591529 3406 heartbeater.cc:464] Registering TS with master...
I20260811 00:25:46.592625 3406 heartbeater.cc:511] Master 127.2.120.62:35005 requested a full tablet report, sending...
I20260811 00:25:46.595446 3186 ts_manager.cc:194] Registered new tserver with Master: cbf74c56200e457c91b84db353142c06 (127.2.120.1:43717)
I20260811 00:25:46.597175 3186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:46385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
I20260811 00:25:46.616883 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:35005
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:46.978978 3415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:46.979279 3415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:46.979378 3415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:46.986819 3415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:46.986963 3415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:46.987082 3415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:25:46.997057 3415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:35005
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:46.998539 3415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:46.999914 3415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:47.015154 3420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:47.015141 3421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:47.015154 3423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:47.016085 3415 server_base.cc:1034] running on GCE node
I20260811 00:25:47.016850 3415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:47.018033 3415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:47.019276 3415 hybrid_clock.cc:648] HybridClock initialized: now 1786407947019221 us; error 44 us; skew 500 ppm
I20260811 00:25:47.019665 3415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:47 dist-test-slave-fd7l krb5kdc[3137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407947, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:47.023188 3415 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:25:47.025120 3415 webserver.cc:460] Webserver started at http://127.2.120.2:37691/ using document root <none> and password file <none>
I20260811 00:25:47.025909 3415 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:47.026074 3415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:47.026417 3415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:47.029860 3415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "130b9bbccc374d8da6ed02bb8faf0e3c"
format_stamp: "Formatted at 2026-08-11 00:25:47 on dist-test-slave-fd7l"
server_key: "c23bf2d4992c939b97550eed92a6a11c"
server_key_iv: "39fe46a53e7343cd99eb3a6614faaf0a"
server_key_version: "encryptionkey@0"
I20260811 00:25:47.030707 3415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "130b9bbccc374d8da6ed02bb8faf0e3c"
format_stamp: "Formatted at 2026-08-11 00:25:47 on dist-test-slave-fd7l"
server_key: "c23bf2d4992c939b97550eed92a6a11c"
server_key_iv: "39fe46a53e7343cd99eb3a6614faaf0a"
server_key_version: "encryptionkey@0"
I20260811 00:25:47.036402 3415 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 00:25:47.040997 3432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:47.042829 3415 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260811 00:25:47.043083 3415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "130b9bbccc374d8da6ed02bb8faf0e3c"
format_stamp: "Formatted at 2026-08-11 00:25:47 on dist-test-slave-fd7l"
server_key: "c23bf2d4992c939b97550eed92a6a11c"
server_key_iv: "39fe46a53e7343cd99eb3a6614faaf0a"
server_key_version: "encryptionkey@0"
I20260811 00:25:47.043334 3415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:47.067628 3415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:47.072407 3415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:47.072765 3415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:47.073794 3415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:25:47.075251 3415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:25:47.075362 3415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:47.075513 3415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:25:47.075585 3415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:47.114061 3415 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:34643
I20260811 00:25:47.114127 3556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:34643 every 8 connection(s)
I20260811 00:25:47.115458 3415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:25:47.120446 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 3415
I20260811 00:25:47.120735 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:25:47.121322 2528 external_mini_cluster.cc:1428] Setting key e811d8feb306b9b1bd7f24c7b88c8b36
Aug 11 00:25:47 dist-test-slave-fd7l krb5kdc[3137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407947, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.120.3@KRBTEST.COM; defaulting to no policy
I20260811 00:25:47.160916 3559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:47.118434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:45703 (local address 127.2.120.62:35005)
0811 00:25:47.118954 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:47.118960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:47.120699 (+ 1739us) server_negotiation.cc:407] Connection header received
0811 00:25:47.122925 (+ 2226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:47.122931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:47.123009 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:47.123135 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:47.126658 (+ 3523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.127484 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.132616 (+ 5132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.133722 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.134244 (+ 522us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:47.144377 (+ 10133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:47.144392 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:47.144401 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:47.144657 (+ 256us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:47.152741 (+ 8084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:47.156004 (+ 3263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:47.156023 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:47.156030 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:47.156840 (+ 810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:47.157950 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:47.157959 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:47.157962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:47.158831 (+ 869us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:47.159109 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:47.160202 (+ 1093us) server_negotiation.cc:299] Negotiation successful
0811 00:25:47.160584 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":281,"threads_started":1}
I20260811 00:25:47.162035 3560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:47.118873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:35005 (local address 127.2.120.2:45703)
0811 00:25:47.119692 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:47.119734 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:47.122607 (+ 2873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:47.123824 (+ 1217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:47.123844 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:47.125005 (+ 1161us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:47.126360 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:47.126413 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.127679 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.127688 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:47.131697 (+ 4009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:47.131765 (+ 68us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.134793 (+ 3028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.134834 (+ 41us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:47.135530 (+ 696us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:47.137734 (+ 2204us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:47.137799 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:47.143414 (+ 5615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:47.153360 (+ 9946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:47.153392 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:47.153451 (+ 59us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:47.155370 (+ 1919us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:47.157363 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:47.157371 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:47.157376 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:47.157761 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:47.159176 (+ 1415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:47.159187 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:47.159686 (+ 499us) client_negotiation.cc:769] Sending connection context
0811 00:25:47.160004 (+ 318us) client_negotiation.cc:240] Negotiation successful
0811 00:25:47.160661 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":319,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.3@KRBTEST.COM" created.
I20260811 00:25:47.163998 3557 heartbeater.cc:347] Connected to a master server at 127.2.120.62:35005
I20260811 00:25:47.164369 3557 heartbeater.cc:464] Registering TS with master...
I20260811 00:25:47.165237 3557 heartbeater.cc:511] Master 127.2.120.62:35005 requested a full tablet report, sending...
I20260811 00:25:47.166879 3186 ts_manager.cc:194] Registered new tserver with Master: 130b9bbccc374d8da6ed02bb8faf0e3c (127.2.120.2:34643)
I20260811 00:25:47.167737 3186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:45703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
I20260811 00:25:47.188598 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:35005
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:47.553113 3563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:47.553493 3563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:47.553665 3563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:47.561407 3563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:47.561579 3563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:47.561776 3563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:25:47.572755 3563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:35005
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:47.574311 3563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:47.575870 3563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:47.592393 3570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:47.592499 3572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:47.592556 3569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:47.593779 3563 server_base.cc:1034] running on GCE node
I20260811 00:25:47.594605 3563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:47.595999 3563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:47.597502 3563 hybrid_clock.cc:648] HybridClock initialized: now 1786407947597388 us; error 94 us; skew 500 ppm
I20260811 00:25:47.597957 3563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:47 dist-test-slave-fd7l krb5kdc[3137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407947, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:47.601696 3406 heartbeater.cc:503] Master 127.2.120.62:35005 was elected leader, sending a full tablet report...
I20260811 00:25:47.601965 3563 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:25:47.603956 3563 webserver.cc:460] Webserver started at http://127.2.120.3:41599/ using document root <none> and password file <none>
I20260811 00:25:47.604874 3563 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:47.605020 3563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:47.605394 3563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:47.608680 3563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "005acd392dc04fbb9ec2d61158e219d0"
format_stamp: "Formatted at 2026-08-11 00:25:47 on dist-test-slave-fd7l"
server_key: "f3ba3b670461572c6c332d5ebb76c4b2"
server_key_iv: "05383ede4d84f2ad18de2adc7d847d86"
server_key_version: "encryptionkey@0"
I20260811 00:25:47.609665 3563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "005acd392dc04fbb9ec2d61158e219d0"
format_stamp: "Formatted at 2026-08-11 00:25:47 on dist-test-slave-fd7l"
server_key: "f3ba3b670461572c6c332d5ebb76c4b2"
server_key_iv: "05383ede4d84f2ad18de2adc7d847d86"
server_key_version: "encryptionkey@0"
I20260811 00:25:47.621162 3563 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.009s sys 0.001s
I20260811 00:25:47.631867 3581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:47.637099 3563 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.008s sys 0.000s
I20260811 00:25:47.637703 3563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "005acd392dc04fbb9ec2d61158e219d0"
format_stamp: "Formatted at 2026-08-11 00:25:47 on dist-test-slave-fd7l"
server_key: "f3ba3b670461572c6c332d5ebb76c4b2"
server_key_iv: "05383ede4d84f2ad18de2adc7d847d86"
server_key_version: "encryptionkey@0"
I20260811 00:25:47.638155 3563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:47.677865 3563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:47.684041 3563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:47.684401 3563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:47.685593 3563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:25:47.687307 3563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:25:47.687391 3563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:47.687549 3563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:25:47.687603 3563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:47.728324 3563 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:37867
I20260811 00:25:47.728341 3707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:37867 every 8 connection(s)
I20260811 00:25:47.730104 3563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:25:47.733916 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 3563
I20260811 00:25:47.734205 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:25:47.734887 2528 external_mini_cluster.cc:1428] Setting key d990114d2e4b7d0646190774915cee98
Aug 11 00:25:47 dist-test-slave-fd7l krb5kdc[3137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407947, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:25:47.760314 3710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:47.734431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:40899 (local address 127.2.120.62:35005)
0811 00:25:47.735111 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:47.735117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:47.736695 (+ 1578us) server_negotiation.cc:407] Connection header received
0811 00:25:47.739459 (+ 2764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:47.739466 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:47.739617 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:47.739767 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:47.742103 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.743011 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.745247 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.745783 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.745910 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:47.750759 (+ 4849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:47.750766 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:47.750770 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:47.750816 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:47.755488 (+ 4672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:47.756721 (+ 1233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:47.756726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:47.756729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:47.756880 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:47.757706 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:47.757710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:47.757712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:47.758055 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:47.758205 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:47.759507 (+ 1302us) server_negotiation.cc:299] Negotiation successful
0811 00:25:47.759831 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":334,"threads_started":1}
I20260811 00:25:47.761531 3711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:47.735183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:35005 (local address 127.2.120.3:40899)
0811 00:25:47.736422 (+ 1239us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:47.736499 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:47.739055 (+ 2556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:47.740204 (+ 1149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:47.740215 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:47.741094 (+ 879us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:47.741855 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:47.741879 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.743229 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.743234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:47.745018 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:47.745038 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.746084 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.746090 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:47.746423 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:47.747542 (+ 1119us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:47.747585 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:47.750395 (+ 2810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:47.755710 (+ 5315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:47.755721 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:47.755739 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:47.756492 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:47.757108 (+ 616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:47.757114 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:47.757117 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:47.757509 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:47.758246 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:47.758255 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:47.758839 (+ 584us) client_negotiation.cc:769] Sending connection context
0811 00:25:47.759218 (+ 379us) client_negotiation.cc:240] Negotiation successful
0811 00:25:47.760009 (+ 791us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":833,"thread_start_us":544,"threads_started":1}
I20260811 00:25:47.763779 3708 heartbeater.cc:347] Connected to a master server at 127.2.120.62:35005
I20260811 00:25:47.764221 3708 heartbeater.cc:464] Registering TS with master...
I20260811 00:25:47.765377 3708 heartbeater.cc:511] Master 127.2.120.62:35005 requested a full tablet report, sending...
I20260811 00:25:47.767628 3186 ts_manager.cc:194] Registered new tserver with Master: 005acd392dc04fbb9ec2d61158e219d0 (127.2.120.3:37867)
I20260811 00:25:47.768800 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:25:47.768865 3186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:40899
I20260811 00:25:47.795933 3710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:47.778707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39126 (local address 127.2.120.62:35005)
0811 00:25:47.778897 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:47.778902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:47.779813 (+ 911us) server_negotiation.cc:407] Connection header received
0811 00:25:47.780086 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:47.780091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:47.780162 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:47.780275 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:47.782069 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.782851 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.784217 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.784723 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.784884 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:47.788563 (+ 3679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:47.788575 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:47.788581 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:47.788650 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:47.791993 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:47.793377 (+ 1384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:47.793389 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:47.793391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:47.793590 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:47.794049 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:47.794057 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:47.794061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:47.794463 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:47.794846 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:47.795200 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 00:25:47.795538 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:25:47.804457 3186 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:25:47.812917 3186 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:25:47.861112 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:47.833838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:37867 (local address 127.0.0.1:50534)
0811 00:25:47.849712 (+ 15874us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:47.849776 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:47.850075 (+ 299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:47.851010 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:47.851017 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:47.851055 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:47.851710 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:47.851728 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.854619 (+ 2891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.854625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:47.857817 (+ 3192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:47.857835 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.859978 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.859984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:47.860210 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:47.860283 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 00:25:47.860440 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 00:25:47.860707 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15501,"thread_start_us":16267,"threads_started":1}
I20260811 00:25:47.861828 3731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:47.834617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50534 (local address 127.2.120.3:37867)
0811 00:25:47.847926 (+ 13309us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:47.847943 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:47.849981 (+ 2038us) server_negotiation.cc:407] Connection header received
0811 00:25:47.850409 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:47.850419 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:47.850726 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:47.850941 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:47.852005 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.854433 (+ 2428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.858068 (+ 3635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.859737 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.859917 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:47.860206 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:47.860555 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 00:25:47.860752 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12334,"thread_start_us":12750,"threads_started":1}
I20260811 00:25:47.862512 3726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:47.834444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43717 (local address 127.0.0.1:45086)
0811 00:25:47.851539 (+ 17095us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:47.851571 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:47.851909 (+ 338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:47.853513 (+ 1604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:47.853518 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:47.853549 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:47.854317 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:47.854371 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.857247 (+ 2876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.857253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:47.859055 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:47.859069 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.861452 (+ 2383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.861458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:47.861651 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:47.861686 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 00:25:47.861782 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 00:25:47.861937 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16803,"spinlock_wait_cycles":16640,"thread_start_us":16122,"threads_started":1}
I20260811 00:25:47.863384 3728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:47.834778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45086 (local address 127.2.120.1:43717)
0811 00:25:47.849127 (+ 14349us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:47.849146 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:47.851747 (+ 2601us) server_negotiation.cc:407] Connection header received
0811 00:25:47.852501 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:47.852598 (+ 97us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:47.853117 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:47.853491 (+ 374us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:47.854613 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.856823 (+ 2210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.859265 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.860876 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.861028 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:47.861286 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:47.862776 (+ 1490us) server_negotiation.cc:299] Negotiation successful
0811 00:25:47.862987 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13921,"thread_start_us":13496,"threads_started":1}
I20260811 00:25:47.864207 3733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:47.843120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59940 (local address 127.2.120.2:34643)
0811 00:25:47.849722 (+ 6602us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:47.849735 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:47.852022 (+ 2287us) server_negotiation.cc:407] Connection header received
0811 00:25:47.852217 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:47.852227 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:47.852652 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:47.852937 (+ 285us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:47.854404 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.856093 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.859520 (+ 3427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.861664 (+ 2144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.861820 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:47.863601 (+ 1781us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:47.863687 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 00:25:47.863861 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6359,"thread_start_us":6148,"threads_started":1}
I20260811 00:25:47.864704 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:47.842781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:34643 (local address 127.0.0.1:59940)
0811 00:25:47.851679 (+ 8898us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:47.851702 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:47.851897 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:47.852941 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:47.852947 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:47.853040 (+ 93us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:47.853861 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:47.853883 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.856333 (+ 2450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.856341 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:47.859271 (+ 2930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:47.859288 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:47.862224 (+ 2936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:47.862231 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:47.862487 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:47.862532 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 00:25:47.862626 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 00:25:47.862885 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8710,"mutex_wait_us":6,"spinlock_wait_cycles":3840,"thread_start_us":6241,"threads_started":1}
I20260811 00:25:47.866532 3634 tablet_service.cc:1467] Processing CreateTablet for tablet 3b77cb8e323d4dd88b1340c9e07cd169 (DEFAULT_TABLE table=test-table [id=1ddd0036749e46c0bf9534572fd27cd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:25:47.867066 3331 tablet_service.cc:1467] Processing CreateTablet for tablet 3b77cb8e323d4dd88b1340c9e07cd169 (DEFAULT_TABLE table=test-table [id=1ddd0036749e46c0bf9534572fd27cd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:25:47.868367 3634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b77cb8e323d4dd88b1340c9e07cd169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:47.868666 3331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b77cb8e323d4dd88b1340c9e07cd169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:47.868791 3484 tablet_service.cc:1467] Processing CreateTablet for tablet 3b77cb8e323d4dd88b1340c9e07cd169 (DEFAULT_TABLE table=test-table [id=1ddd0036749e46c0bf9534572fd27cd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:25:47.870258 3484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b77cb8e323d4dd88b1340c9e07cd169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:47.883986 3734 tablet_bootstrap.cc:492] T 3b77cb8e323d4dd88b1340c9e07cd169 P cbf74c56200e457c91b84db353142c06: Bootstrap starting.
I20260811 00:25:47.887976 3734 tablet_bootstrap.cc:654] T 3b77cb8e323d4dd88b1340c9e07cd169 P cbf74c56200e457c91b84db353142c06: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:47.888377 3736 tablet_bootstrap.cc:492] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c: Bootstrap starting.
I20260811 00:25:47.889863 3734 log.cc:824] T 3b77cb8e323d4dd88b1340c9e07cd169 P cbf74c56200e457c91b84db353142c06: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:47.890619 3735 tablet_bootstrap.cc:492] T 3b77cb8e323d4dd88b1340c9e07cd169 P 005acd392dc04fbb9ec2d61158e219d0: Bootstrap starting.
I20260811 00:25:47.894098 3736 tablet_bootstrap.cc:654] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:47.894140 3735 tablet_bootstrap.cc:654] T 3b77cb8e323d4dd88b1340c9e07cd169 P 005acd392dc04fbb9ec2d61158e219d0: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:47.895046 3734 tablet_bootstrap.cc:492] T 3b77cb8e323d4dd88b1340c9e07cd169 P cbf74c56200e457c91b84db353142c06: No bootstrap required, opened a new log
I20260811 00:25:47.895421 3734 ts_tablet_manager.cc:1397] T 3b77cb8e323d4dd88b1340c9e07cd169 P cbf74c56200e457c91b84db353142c06: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260811 00:25:47.895452 3735 log.cc:824] T 3b77cb8e323d4dd88b1340c9e07cd169 P 005acd392dc04fbb9ec2d61158e219d0: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:47.895658 3736 log.cc:824] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:47.898910 3736 tablet_bootstrap.cc:492] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c: No bootstrap required, opened a new log
I20260811 00:25:47.899236 3735 tablet_bootstrap.cc:492] T 3b77cb8e323d4dd88b1340c9e07cd169 P 005acd392dc04fbb9ec2d61158e219d0: No bootstrap required, opened a new log
I20260811 00:25:47.899248 3736 ts_tablet_manager.cc:1397] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260811 00:25:47.899546 3735 ts_tablet_manager.cc:1397] T 3b77cb8e323d4dd88b1340c9e07cd169 P 005acd392dc04fbb9ec2d61158e219d0: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 00:25:47.907389 3736 raft_consensus.cc:348] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005acd392dc04fbb9ec2d61158e219d0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37867 } } peers { permanent_uuid: "cbf74c56200e457c91b84db353142c06" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43717 } } peers { permanent_uuid: "130b9bbccc374d8da6ed02bb8faf0e3c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 34643 } }
I20260811 00:25:47.907414 3734 raft_consensus.cc:348] T 3b77cb8e323d4dd88b1340c9e07cd169 P cbf74c56200e457c91b84db353142c06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005acd392dc04fbb9ec2d61158e219d0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37867 } } peers { permanent_uuid: "cbf74c56200e457c91b84db353142c06" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43717 } } peers { permanent_uuid: "130b9bbccc374d8da6ed02bb8faf0e3c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 34643 } }
I20260811 00:25:47.907855 3736 raft_consensus.cc:374] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:47.907900 3734 raft_consensus.cc:374] T 3b77cb8e323d4dd88b1340c9e07cd169 P cbf74c56200e457c91b84db353142c06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:47.907976 3736 raft_consensus.cc:729] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130b9bbccc374d8da6ed02bb8faf0e3c, State: Initialized, Role: FOLLOWER
I20260811 00:25:47.907996 3734 raft_consensus.cc:729] T 3b77cb8e323d4dd88b1340c9e07cd169 P cbf74c56200e457c91b84db353142c06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf74c56200e457c91b84db353142c06, State: Initialized, Role: FOLLOWER
I20260811 00:25:47.908736 3736 consensus_queue.cc:260] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005acd392dc04fbb9ec2d61158e219d0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37867 } } peers { permanent_uuid: "cbf74c56200e457c91b84db353142c06" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43717 } } peers { permanent_uuid: "130b9bbccc374d8da6ed02bb8faf0e3c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 34643 } }
I20260811 00:25:47.908790 3734 consensus_queue.cc:260] T 3b77cb8e323d4dd88b1340c9e07cd169 P cbf74c56200e457c91b84db353142c06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005acd392dc04fbb9ec2d61158e219d0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37867 } } peers { permanent_uuid: "cbf74c56200e457c91b84db353142c06" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43717 } } peers { permanent_uuid: "130b9bbccc374d8da6ed02bb8faf0e3c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 34643 } }
I20260811 00:25:47.909994 3557 heartbeater.cc:503] Master 127.2.120.62:35005 was elected leader, sending a full tablet report...
I20260811 00:25:47.911001 3736 ts_tablet_manager.cc:1428] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260811 00:25:47.912194 3735 raft_consensus.cc:348] T 3b77cb8e323d4dd88b1340c9e07cd169 P 005acd392dc04fbb9ec2d61158e219d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005acd392dc04fbb9ec2d61158e219d0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37867 } } peers { permanent_uuid: "cbf74c56200e457c91b84db353142c06" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43717 } } peers { permanent_uuid: "130b9bbccc374d8da6ed02bb8faf0e3c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 34643 } }
I20260811 00:25:47.912763 3735 raft_consensus.cc:374] T 3b77cb8e323d4dd88b1340c9e07cd169 P 005acd392dc04fbb9ec2d61158e219d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:47.912873 3735 raft_consensus.cc:729] T 3b77cb8e323d4dd88b1340c9e07cd169 P 005acd392dc04fbb9ec2d61158e219d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 005acd392dc04fbb9ec2d61158e219d0, State: Initialized, Role: FOLLOWER
I20260811 00:25:47.913297 3734 ts_tablet_manager.cc:1428] T 3b77cb8e323d4dd88b1340c9e07cd169 P cbf74c56200e457c91b84db353142c06: Time spent starting tablet: real 0.018s user 0.020s sys 0.000s
I20260811 00:25:47.913686 3735 consensus_queue.cc:260] T 3b77cb8e323d4dd88b1340c9e07cd169 P 005acd392dc04fbb9ec2d61158e219d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005acd392dc04fbb9ec2d61158e219d0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37867 } } peers { permanent_uuid: "cbf74c56200e457c91b84db353142c06" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43717 } } peers { permanent_uuid: "130b9bbccc374d8da6ed02bb8faf0e3c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 34643 } }
I20260811 00:25:47.917533 3708 heartbeater.cc:503] Master 127.2.120.62:35005 was elected leader, sending a full tablet report...
I20260811 00:25:47.918331 3735 ts_tablet_manager.cc:1428] T 3b77cb8e323d4dd88b1340c9e07cd169 P 005acd392dc04fbb9ec2d61158e219d0: Time spent starting tablet: real 0.019s user 0.015s sys 0.003s
W20260811 00:25:47.982282 3709 tablet.cc:2369] T 3b77cb8e323d4dd88b1340c9e07cd169 P 005acd392dc04fbb9ec2d61158e219d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:25:48.002558 3740 raft_consensus.cc:482] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:25:48.003098 3740 raft_consensus.cc:504] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005acd392dc04fbb9ec2d61158e219d0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37867 } } peers { permanent_uuid: "cbf74c56200e457c91b84db353142c06" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43717 } } peers { permanent_uuid: "130b9bbccc374d8da6ed02bb8faf0e3c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 34643 } }
I20260811 00:25:48.005439 3740 leader_election.cc:288] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 005acd392dc04fbb9ec2d61158e219d0 (127.2.120.3:37867), cbf74c56200e457c91b84db353142c06 (127.2.120.1:43717)
I20260811 00:25:48.013999 3731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:48.006091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:35079 (local address 127.2.120.3:37867)
0811 00:25:48.006294 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:48.006302 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:48.007101 (+ 799us) server_negotiation.cc:407] Connection header received
0811 00:25:48.007554 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:48.007563 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:48.007677 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:48.007849 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:48.008918 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.010105 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.012007 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.012949 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.013069 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:48.013167 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:48.013581 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 00:25:48.013696 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 00:25:48.014135 3744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:48.005922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:37867 (local address 127.2.120.2:35079)
0811 00:25:48.006863 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:48.006879 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:48.007371 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:48.008084 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:48.008089 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:48.008119 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:48.008714 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:48.008731 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.010245 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.010250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:48.011797 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:48.011812 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.013206 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.013214 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:48.013388 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:48.013425 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 00:25:48.013533 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 00:25:48.013701 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"mutex_wait_us":21,"spinlock_wait_cycles":790912,"thread_start_us":556,"threads_started":1}
I20260811 00:25:48.015744 3657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b77cb8e323d4dd88b1340c9e07cd169" candidate_uuid: "130b9bbccc374d8da6ed02bb8faf0e3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "005acd392dc04fbb9ec2d61158e219d0" is_pre_election: true
I20260811 00:25:48.016328 3728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:48.006121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:46325 (local address 127.2.120.1:43717)
0811 00:25:48.006395 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:48.006402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:48.007105 (+ 703us) server_negotiation.cc:407] Connection header received
0811 00:25:48.007588 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:48.007592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:48.007697 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:48.007787 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:48.008757 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.010105 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.013902 (+ 3797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.014992 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.015114 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:48.015235 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:48.015886 (+ 651us) server_negotiation.cc:299] Negotiation successful
0811 00:25:48.016077 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260811 00:25:48.016438 3657 raft_consensus.cc:2454] T 3b77cb8e323d4dd88b1340c9e07cd169 P 005acd392dc04fbb9ec2d61158e219d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130b9bbccc374d8da6ed02bb8faf0e3c in term 0.
I20260811 00:25:48.016587 3743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:48.005859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43717 (local address 127.2.120.2:46325)
0811 00:25:48.006785 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:48.006807 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:48.007371 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:48.007860 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:48.007865 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:48.007896 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:48.008590 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:48.008612 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.011957 (+ 3345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.011961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:48.013694 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:48.013707 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.015352 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.015360 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:48.015555 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:48.015593 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 00:25:48.015864 (+ 271us) client_negotiation.cc:240] Negotiation successful
0811 00:25:48.016011 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":48,"spinlock_wait_cycles":2304,"thread_start_us":454,"threads_started":1}
I20260811 00:25:48.017421 3434 leader_election.cc:302] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 005acd392dc04fbb9ec2d61158e219d0, 130b9bbccc374d8da6ed02bb8faf0e3c; no voters:
I20260811 00:25:48.017967 3740 raft_consensus.cc:2781] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:25:48.018253 3740 raft_consensus.cc:482] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:25:48.018392 3740 raft_consensus.cc:3037] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:48.018220 3353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b77cb8e323d4dd88b1340c9e07cd169" candidate_uuid: "130b9bbccc374d8da6ed02bb8faf0e3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf74c56200e457c91b84db353142c06" is_pre_election: true
I20260811 00:25:48.019071 3353 raft_consensus.cc:2454] T 3b77cb8e323d4dd88b1340c9e07cd169 P cbf74c56200e457c91b84db353142c06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 130b9bbccc374d8da6ed02bb8faf0e3c in term 0.
I20260811 00:25:48.021792 3740 raft_consensus.cc:504] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005acd392dc04fbb9ec2d61158e219d0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37867 } } peers { permanent_uuid: "cbf74c56200e457c91b84db353142c06" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43717 } } peers { permanent_uuid: "130b9bbccc374d8da6ed02bb8faf0e3c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 34643 } }
I20260811 00:25:48.022529 3740 leader_election.cc:288] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c [CANDIDATE]: Term 1 election: Requested vote from peers 005acd392dc04fbb9ec2d61158e219d0 (127.2.120.3:37867), cbf74c56200e457c91b84db353142c06 (127.2.120.1:43717)
I20260811 00:25:48.023211 3657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b77cb8e323d4dd88b1340c9e07cd169" candidate_uuid: "130b9bbccc374d8da6ed02bb8faf0e3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "005acd392dc04fbb9ec2d61158e219d0"
I20260811 00:25:48.023417 3657 raft_consensus.cc:3037] T 3b77cb8e323d4dd88b1340c9e07cd169 P 005acd392dc04fbb9ec2d61158e219d0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:48.023494 3353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b77cb8e323d4dd88b1340c9e07cd169" candidate_uuid: "130b9bbccc374d8da6ed02bb8faf0e3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf74c56200e457c91b84db353142c06"
I20260811 00:25:48.023901 3353 raft_consensus.cc:3037] T 3b77cb8e323d4dd88b1340c9e07cd169 P cbf74c56200e457c91b84db353142c06 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:48.026263 3657 raft_consensus.cc:2454] T 3b77cb8e323d4dd88b1340c9e07cd169 P 005acd392dc04fbb9ec2d61158e219d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130b9bbccc374d8da6ed02bb8faf0e3c in term 1.
I20260811 00:25:48.027009 3434 leader_election.cc:302] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 005acd392dc04fbb9ec2d61158e219d0, 130b9bbccc374d8da6ed02bb8faf0e3c; no voters:
I20260811 00:25:48.027565 3740 raft_consensus.cc:2781] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:48.027740 3353 raft_consensus.cc:2454] T 3b77cb8e323d4dd88b1340c9e07cd169 P cbf74c56200e457c91b84db353142c06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 130b9bbccc374d8da6ed02bb8faf0e3c in term 1.
I20260811 00:25:48.028381 3740 raft_consensus.cc:686] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c [term 1 LEADER]: Becoming Leader. State: Replica: 130b9bbccc374d8da6ed02bb8faf0e3c, State: Running, Role: LEADER
I20260811 00:25:48.029304 3740 consensus_queue.cc:237] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005acd392dc04fbb9ec2d61158e219d0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37867 } } peers { permanent_uuid: "cbf74c56200e457c91b84db353142c06" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43717 } } peers { permanent_uuid: "130b9bbccc374d8da6ed02bb8faf0e3c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 34643 } }
I20260811 00:25:48.036038 3186 catalog_manager.cc:5495] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 130b9bbccc374d8da6ed02bb8faf0e3c (127.2.120.2). New cstate: current_term: 1 leader_uuid: "130b9bbccc374d8da6ed02bb8faf0e3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005acd392dc04fbb9ec2d61158e219d0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbf74c56200e457c91b84db353142c06" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "130b9bbccc374d8da6ed02bb8faf0e3c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 34643 } health_report { overall_health: HEALTHY } } }
W20260811 00:25:48.051901 3407 tablet.cc:2369] T 3b77cb8e323d4dd88b1340c9e07cd169 P cbf74c56200e457c91b84db353142c06: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 00:25:48 dist-test-slave-fd7l krb5kdc[3137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:25:48.068497 3728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:48.049642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45100 (local address 127.2.120.1:43717)
0811 00:25:48.049906 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:48.049916 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:48.050701 (+ 785us) server_negotiation.cc:407] Connection header received
0811 00:25:48.050921 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:48.050926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:48.051061 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:48.051178 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:48.053405 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.054547 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.056128 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.056734 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.056873 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:48.061688 (+ 4815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:48.061708 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:48.061728 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:48.061776 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:48.065226 (+ 3450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:48.066099 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:48.066107 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:48.066109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:48.066279 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:48.066704 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:48.066710 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:48.066712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:48.067188 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:48.067606 (+ 418us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:48.067747 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 00:25:48.068048 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 00:25:48 dist-test-slave-fd7l krb5kdc[3137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407948, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 00:25:48 dist-test-slave-fd7l krb5kdc[3137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407948, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
W20260811 00:25:48.119434 3558 tablet.cc:2369] T 3b77cb8e323d4dd88b1340c9e07cd169 P 130b9bbccc374d8da6ed02bb8faf0e3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:25:48.126811 3728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:48.109908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45116 (local address 127.2.120.1:43717)
0811 00:25:48.110412 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:48.110420 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:48.110772 (+ 352us) server_negotiation.cc:407] Connection header received
0811 00:25:48.111006 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:48.111010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:48.111085 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:48.111199 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:48.112857 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.113686 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.115045 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.115483 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.115647 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:48.120653 (+ 5006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:48.120659 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:48.120662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:48.120700 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:48.123573 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:48.124445 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:48.124457 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:48.124460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:48.124692 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:48.125202 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:48.125206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:48.125208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:48.125494 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:48.125715 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:48.126077 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 00:25:48.126344 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224}
Aug 11 00:25:48 dist-test-slave-fd7l krb5kdc[3137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 00:25:48 dist-test-slave-fd7l krb5kdc[3137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:25:48.183234 3728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:48.167269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45132 (local address 127.2.120.1:43717)
0811 00:25:48.167561 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:48.167568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:48.167932 (+ 364us) server_negotiation.cc:407] Connection header received
0811 00:25:48.168115 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:48.168119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:48.168200 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:48.168336 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:48.170060 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.170781 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.172216 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.172738 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.172842 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:48.177234 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:48.177240 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:48.177243 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:48.177285 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:48.180073 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:48.180928 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:48.180932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:48.180935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:48.181059 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:48.181428 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:48.181431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:48.181433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:48.181817 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:48.182045 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:48.182414 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 00:25:48.182739 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
W20260811 00:25:48.185773 3307 tablet_service.cc:2908] Rejecting scan request for tablet 3b77cb8e323d4dd88b1340c9e07cd169: Uninitialized: clean time has not yet been initialized
I20260811 00:25:48.189541 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 3256
I20260811 00:25:48.204181 3728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:48.190565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45146 (local address 127.2.120.1:43717)
0811 00:25:48.190799 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:48.190808 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:48.191132 (+ 324us) server_negotiation.cc:407] Connection header received
0811 00:25:48.191232 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:48.191237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:48.191305 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:48.191405 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:48.192640 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.193424 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.194617 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.195110 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.195231 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:48.198036 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:48.198048 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:48.198051 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:48.198100 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:48.201107 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:48.201975 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:48.201981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:48.201983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:48.202160 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:48.202617 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:48.202621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:48.202623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:48.202947 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:48.203365 (+ 418us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:48.203524 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 00:25:48.203728 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 00:25:48.205041 3399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:25:48.303202 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 3256
I20260811 00:25:48.327070 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 3415
Aug 11 00:25:48 dist-test-slave-fd7l krb5kdc[3137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.2@KRBTEST.COM
I20260811 00:25:48.343367 3733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:48.327993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59942 (local address 127.2.120.2:34643)
0811 00:25:48.328189 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:48.328194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:48.328212 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:25:48.328271 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:48.328275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:48.328338 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:48.328431 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:48.329852 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.330703 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.332155 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.332660 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.332769 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:48.337324 (+ 4555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:48.337344 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:48.337364 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:48.337411 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:48.340463 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:48.341307 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:48.341314 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:48.341317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:48.341489 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:48.341861 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:48.341864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:48.341866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:48.342270 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:48.342677 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:48.342878 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 00:25:48.343099 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:25:48.344308 3550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:25:48.438675 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 3415
I20260811 00:25:48.438802 3743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:48.438199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43717 (local address 127.2.120.2:47489)
0811 00:25:48.438513 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:48.438629 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
W20260811 00:25:48.476258 3584 connection.cc:535] server connection from 127.2.120.2:35079 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 00:25:48.476943 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 3563
Aug 11 00:25:48 dist-test-slave-fd7l krb5kdc[3137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.3@KRBTEST.COM
I20260811 00:25:48.497102 3731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:48.479272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50548 (local address 127.2.120.3:37867)
0811 00:25:48.479593 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:48.479601 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:48.479745 (+ 144us) server_negotiation.cc:407] Connection header received
0811 00:25:48.480090 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:48.480096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:48.480209 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:48.480369 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:48.482109 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.482927 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.484625 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:48.485147 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:48.485311 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:48.490601 (+ 5290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:48.490621 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:48.490640 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:48.490713 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:48.493819 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:48.494678 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:48.494685 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:48.494688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:48.494848 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:48.495301 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:48.495305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:48.495307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:48.495720 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:48.496039 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:48.496409 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 00:25:48.496714 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260811 00:25:48.498011 3703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:25:48.591203 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 3563
I20260811 00:25:48.613699 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 3151
I20260811 00:25:48.615317 3217 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:25:48.728681 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 3151
2026-08-11T00:25:48Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4841 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:25:48 dist-test-slave-fd7l krb5kdc[3764](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:25:48 dist-test-slave-fd7l krb5kdc[3764](info): set up 2 sockets
Aug 11 00:25:48 dist-test-slave-fd7l krb5kdc[3764](info): commencing operation
krb5kdc: starting...
W20260811 00:25:49.961961 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:25:50 dist-test-slave-fd7l krb5kdc[3764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:25:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:25:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:25:50.201277 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:43815
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:43815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:50.587898 3777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:50.588419 3777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:50.588660 3777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:50.596980 3777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:25:50.597126 3777 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 00:25:50.597174 3777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:50.597246 3777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:50.597306 3777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:25:50.597383 3777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:25:50.609256 3777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:43815
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:43815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:50.610704 3777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:50.612066 3777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:50.624881 3783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:50.624925 3786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:50.624956 3784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:50.625975 3777 server_base.cc:1034] running on GCE node
I20260811 00:25:50.626868 3777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:50.628582 3777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:50.629833 3777 hybrid_clock.cc:648] HybridClock initialized: now 1786407950629765 us; error 44 us; skew 500 ppm
I20260811 00:25:50.630275 3777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:50 dist-test-slave-fd7l krb5kdc[3764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407950, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:50.633910 3777 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:25:50.635635 3777 webserver.cc:460] Webserver started at http://127.2.120.62:39905/ using document root <none> and password file <none>
I20260811 00:25:50.636472 3777 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:50.636694 3777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:50.637084 3777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:50.640323 3777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "b90fe2e2cb494eceb5c50925693b0858"
format_stamp: "Formatted at 2026-08-11 00:25:50 on dist-test-slave-fd7l"
server_key: "22d0e2882461e06cf5960d3ddae4c609"
server_key_iv: "62fc7357ae32acb0be136c5374c4b900"
server_key_version: "encryptionkey@0"
I20260811 00:25:50.641311 3777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "b90fe2e2cb494eceb5c50925693b0858"
format_stamp: "Formatted at 2026-08-11 00:25:50 on dist-test-slave-fd7l"
server_key: "22d0e2882461e06cf5960d3ddae4c609"
server_key_iv: "62fc7357ae32acb0be136c5374c4b900"
server_key_version: "encryptionkey@0"
I20260811 00:25:50.647447 3777 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:25:50.651577 3793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:50.653380 3777 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:25:50.653566 3777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "b90fe2e2cb494eceb5c50925693b0858"
format_stamp: "Formatted at 2026-08-11 00:25:50 on dist-test-slave-fd7l"
server_key: "22d0e2882461e06cf5960d3ddae4c609"
server_key_iv: "62fc7357ae32acb0be136c5374c4b900"
server_key_version: "encryptionkey@0"
I20260811 00:25:50.653797 3777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:50.689301 3777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:50.693955 3777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:50.694279 3777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:50.718036 3777 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:43815
I20260811 00:25:50.718089 3865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:43815 every 8 connection(s)
I20260811 00:25:50.719460 3777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:25:50.723780 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:50.725358 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 3777
I20260811 00:25:50.725625 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:25:50.726608 2528 external_mini_cluster.cc:1428] Setting key 08fac8a20e4bca46dfbc2717f0ceec23
I20260811 00:25:50.740221 3867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858: Bootstrap starting.
I20260811 00:25:50.743817 3867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:50.745182 3867 log.cc:824] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:25:50 dist-test-slave-fd7l krb5kdc[3764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:25:50.748661 3867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858: No bootstrap required, opened a new log
I20260811 00:25:50.757074 3871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:50.729167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41382 (local address 127.2.120.62:43815)
0811 00:25:50.730150 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:50.730163 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:50.730211 (+ 48us) server_negotiation.cc:407] Connection header received
0811 00:25:50.731955 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:50.731983 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:50.732470 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:50.733067 (+ 597us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:50.734704 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:50.736025 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:50.737713 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:50.738486 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:50.738709 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:50.748626 (+ 9917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:50.748637 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:50.748658 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:50.748728 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:50.752121 (+ 3393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:50.753060 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:50.753069 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:50.753076 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:50.753260 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:50.753698 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:50.753711 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:50.753713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:50.754342 (+ 629us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:50.754721 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:50.755174 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 00:25:50.755793 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":488,"threads_started":1}
I20260811 00:25:50.758502 3867 raft_consensus.cc:348] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90fe2e2cb494eceb5c50925693b0858" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43815 } }
I20260811 00:25:50.759025 3867 raft_consensus.cc:374] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:50.759163 3867 raft_consensus.cc:729] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b90fe2e2cb494eceb5c50925693b0858, State: Initialized, Role: FOLLOWER
I20260811 00:25:50.759831 3867 consensus_queue.cc:260] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90fe2e2cb494eceb5c50925693b0858" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43815 } }
I20260811 00:25:50.760125 3867 raft_consensus.cc:388] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:25:50.760444 3867 raft_consensus.cc:482] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:25:50.760689 3867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:50.763087 3867 raft_consensus.cc:504] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90fe2e2cb494eceb5c50925693b0858" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43815 } }
I20260811 00:25:50.763681 3867 leader_election.cc:302] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b90fe2e2cb494eceb5c50925693b0858; no voters:
I20260811 00:25:50.764299 3867 leader_election.cc:288] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:25:50.764444 3874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:50.765228 3874 raft_consensus.cc:686] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858 [term 1 LEADER]: Becoming Leader. State: Replica: b90fe2e2cb494eceb5c50925693b0858, State: Running, Role: LEADER
I20260811 00:25:50.765838 3874 consensus_queue.cc:237] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90fe2e2cb494eceb5c50925693b0858" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43815 } }
I20260811 00:25:50.766718 3867 sys_catalog.cc:564] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:25:50.769174 3876 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b90fe2e2cb494eceb5c50925693b0858. Latest consensus state: current_term: 1 leader_uuid: "b90fe2e2cb494eceb5c50925693b0858" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90fe2e2cb494eceb5c50925693b0858" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43815 } } }
I20260811 00:25:50.769688 3876 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:50.769201 3875 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b90fe2e2cb494eceb5c50925693b0858" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90fe2e2cb494eceb5c50925693b0858" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43815 } } }
I20260811 00:25:50.769858 3875 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858 [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:50.773171 3880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:25:50.778224 3880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:25:50.794757 3880 catalog_manager.cc:1347] Generated new cluster ID: 0e29a2a6826a4dde80681dd0bb5072a4
I20260811 00:25:50.794906 3880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:25:50.816710 3880 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:25:50.818910 3880 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:25:50.829949 3880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858: Generated new TSK 0
I20260811 00:25:50.831100 3880 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
I20260811 00:25:50.882731 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43815
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:51.262542 3896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:51.263104 3896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:51.263315 3896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:51.274194 3896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:51.274505 3896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:51.274693 3896 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:25:51.274850 3896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:25:51.288326 3896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.120.62:43815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:51.289942 3896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:51.291498 3896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:51.306996 3902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:51.306931 3903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:51.307029 3905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:51.308353 3896 server_base.cc:1034] running on GCE node
I20260811 00:25:51.309156 3896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:51.310441 3896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:51.311889 3896 hybrid_clock.cc:648] HybridClock initialized: now 1786407951311741 us; error 77 us; skew 500 ppm
I20260811 00:25:51.312642 3896 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:51 dist-test-slave-fd7l krb5kdc[3764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407951, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:51.316463 3896 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:25:51.318243 3896 webserver.cc:460] Webserver started at http://127.2.120.1:34891/ using document root <none> and password file <none>
I20260811 00:25:51.319068 3896 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:51.319219 3896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:51.319622 3896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:51.323697 3896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0"
format_stamp: "Formatted at 2026-08-11 00:25:51 on dist-test-slave-fd7l"
server_key: "a5098f5b604b2832834716ec3c772270"
server_key_iv: "2d4796aa29f44eeeca1598746b73d3fa"
server_key_version: "encryptionkey@0"
I20260811 00:25:51.324756 3896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0"
format_stamp: "Formatted at 2026-08-11 00:25:51 on dist-test-slave-fd7l"
server_key: "a5098f5b604b2832834716ec3c772270"
server_key_iv: "2d4796aa29f44eeeca1598746b73d3fa"
server_key_version: "encryptionkey@0"
I20260811 00:25:51.330976 3896 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:25:51.334964 3913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:51.336447 3896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:25:51.336720 3896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0"
format_stamp: "Formatted at 2026-08-11 00:25:51 on dist-test-slave-fd7l"
server_key: "a5098f5b604b2832834716ec3c772270"
server_key_iv: "2d4796aa29f44eeeca1598746b73d3fa"
server_key_version: "encryptionkey@0"
I20260811 00:25:51.336977 3896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:51.353238 3896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:51.358210 3896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:51.358558 3896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:51.361723 3896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:25:51.361853 3896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:51.362032 3896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:25:51.362113 3896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:25:51 dist-test-slave-fd7l krb5kdc[3764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407951, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:25:51.389843 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:51.362232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:36407 (local address 127.2.120.62:43815)
0811 00:25:51.363193 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:51.363201 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:51.364361 (+ 1160us) server_negotiation.cc:407] Connection header received
0811 00:25:51.366846 (+ 2485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:51.366852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:51.366984 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:51.367137 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:51.370181 (+ 3044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:51.371588 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:51.374510 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:51.375252 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:51.375463 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:51.381433 (+ 5970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:51.381439 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:51.381443 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:51.381487 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:51.384666 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:51.386255 (+ 1589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:51.386268 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:51.386270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:51.386501 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:51.387191 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:51.387198 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:51.387202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:51.387657 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:51.387853 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:51.389102 (+ 1249us) server_negotiation.cc:299] Negotiation successful
0811 00:25:51.389408 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":581,"threads_started":1}
I20260811 00:25:51.391013 3923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:51.362869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:43815 (local address 127.2.120.1:36407)
0811 00:25:51.364136 (+ 1267us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:51.364191 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:51.366397 (+ 2206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:51.367495 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:51.367514 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:51.368654 (+ 1140us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:51.369847 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:51.369892 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:51.371811 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:51.371825 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:51.374274 (+ 2449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:51.374298 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:51.375608 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:51.375620 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:51.376042 (+ 422us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:51.377076 (+ 1034us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:51.377117 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:51.381098 (+ 3981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:51.384911 (+ 3813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:51.384924 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:51.384956 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:51.386002 (+ 1046us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:51.386709 (+ 707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:51.386717 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:51.386725 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:51.387024 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:51.387962 (+ 938us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:51.387972 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:51.388489 (+ 517us) client_negotiation.cc:769] Sending connection context
0811 00:25:51.388955 (+ 466us) client_negotiation.cc:240] Negotiation successful
0811 00:25:51.389600 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":910,"thread_start_us":502,"threads_started":1}
I20260811 00:25:51.416611 3922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:51.398494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:35701 (local address 127.2.120.62:43815)
0811 00:25:51.398768 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:51.398777 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:51.399526 (+ 749us) server_negotiation.cc:407] Connection header received
0811 00:25:51.399908 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:51.399916 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:51.400020 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:51.400120 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:51.402266 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:51.403547 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:51.406402 (+ 2855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:51.407038 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:51.407215 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:51.409680 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:51.409686 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:51.409689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:51.409726 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:51.413070 (+ 3344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:51.413900 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:51.413909 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:51.413915 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:51.414103 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:51.414655 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:51.414661 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:51.414665 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:51.415114 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:51.415297 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:51.415887 (+ 590us) server_negotiation.cc:299] Negotiation successful
0811 00:25:51.416134 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 00:25:51.416764 3987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:51.398490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:43815 (local address 127.2.120.1:35701)
0811 00:25:51.399395 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:51.399424 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:51.399703 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:51.400411 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:51.400418 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:51.401416 (+ 998us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:51.401979 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:51.402001 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:51.403781 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:51.403790 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:51.406179 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:51.406206 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:51.407355 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:51.407364 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:51.407536 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:51.408431 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:51.408461 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:51.409443 (+ 982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:51.413227 (+ 3784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:51.413233 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:51.413238 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:51.413750 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:51.414271 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:51.414283 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:51.414285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:51.414468 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:51.415303 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:51.415310 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:51.415557 (+ 247us) client_negotiation.cc:769] Sending connection context
0811 00:25:51.415752 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 00:25:51.416134 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":482,"threads_started":1}
I20260811 00:25:51.423241 3896 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:39645
I20260811 00:25:51.423252 4046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:39645 every 8 connection(s)
I20260811 00:25:51.424592 3896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:25:51.426690 4047 heartbeater.cc:347] Connected to a master server at 127.2.120.62:43815
I20260811 00:25:51.426937 4047 heartbeater.cc:464] Registering TS with master...
I20260811 00:25:51.427660 4047 heartbeater.cc:511] Master 127.2.120.62:43815 requested a full tablet report, sending...
I20260811 00:25:51.429466 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 3896
I20260811 00:25:51.429733 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:25:51.430408 3816 ts_manager.cc:194] Registered new tserver with Master: 7ba1bd2b484e4d648a76ef97a1fcdbb0 (127.2.120.1:39645)
I20260811 00:25:51.430691 2528 external_mini_cluster.cc:1428] Setting key 8f23a5714a610218a96d3cc6165d085a
I20260811 00:25:51.432601 3816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:36407
WARNING: no policy specified for kudu/127.2.120.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
I20260811 00:25:51.484083 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43815
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:51.841193 4052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:51.841559 4052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:51.841754 4052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:51.849280 4052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:51.849406 4052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:51.849556 4052 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:25:51.849709 4052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:25:51.860028 4052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.120.62:43815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:51.861423 4052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:51.862848 4052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:51.877605 4058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:51.877581 4061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:51.877632 4059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:51.878868 4052 server_base.cc:1034] running on GCE node
I20260811 00:25:51.879678 4052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:51.880833 4052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:51.882144 4052 hybrid_clock.cc:648] HybridClock initialized: now 1786407951882067 us; error 44 us; skew 500 ppm
I20260811 00:25:51.882513 4052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:51 dist-test-slave-fd7l krb5kdc[3764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407951, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:51.886569 4052 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:25:51.888161 4052 webserver.cc:460] Webserver started at http://127.2.120.2:44713/ using document root <none> and password file <none>
I20260811 00:25:51.888985 4052 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:51.889092 4052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:51.889385 4052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:51.892477 4052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "95af07ebb47248a6af9fa492eef17972"
format_stamp: "Formatted at 2026-08-11 00:25:51 on dist-test-slave-fd7l"
server_key: "f028c9a4459e52b15f0489cb3c4e8c2f"
server_key_iv: "4d2c8a801f9968c939d5570dde542443"
server_key_version: "encryptionkey@0"
I20260811 00:25:51.893460 4052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "95af07ebb47248a6af9fa492eef17972"
format_stamp: "Formatted at 2026-08-11 00:25:51 on dist-test-slave-fd7l"
server_key: "f028c9a4459e52b15f0489cb3c4e8c2f"
server_key_iv: "4d2c8a801f9968c939d5570dde542443"
server_key_version: "encryptionkey@0"
I20260811 00:25:51.899077 4052 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 00:25:51.903085 4069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:51.904625 4052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:25:51.904824 4052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "95af07ebb47248a6af9fa492eef17972"
format_stamp: "Formatted at 2026-08-11 00:25:51 on dist-test-slave-fd7l"
server_key: "f028c9a4459e52b15f0489cb3c4e8c2f"
server_key_iv: "4d2c8a801f9968c939d5570dde542443"
server_key_version: "encryptionkey@0"
I20260811 00:25:51.905077 4052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:51.926726 4052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:51.931690 4052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:51.931999 4052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:51.934964 4052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:25:51.935055 4052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:51.935262 4052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:25:51.935326 4052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:25:51 dist-test-slave-fd7l krb5kdc[3764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407951, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:25:51.966988 4078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:51.935917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:41981 (local address 127.2.120.62:43815)
0811 00:25:51.936453 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:51.936460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:51.937963 (+ 1503us) server_negotiation.cc:407] Connection header received
0811 00:25:51.941216 (+ 3253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:51.941223 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:51.941339 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:51.941487 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:51.944599 (+ 3112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:51.945643 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:51.948302 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:51.950561 (+ 2259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:51.950766 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:51.957582 (+ 6816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:51.957593 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:51.957601 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:51.957668 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:51.961936 (+ 4268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:51.963316 (+ 1380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:51.963325 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:51.963329 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:51.963492 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:51.964240 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:51.964247 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:51.964250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:51.964787 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:51.964973 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:51.966081 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0811 00:25:51.966490 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":267,"threads_started":1}
I20260811 00:25:51.968762 4079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:51.936663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:43815 (local address 127.2.120.2:41981)
0811 00:25:51.937760 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:51.937817 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:51.940839 (+ 3022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:51.941867 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:51.941883 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:51.942939 (+ 1056us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:51.944234 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:51.944271 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:51.945941 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:51.945949 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:51.948079 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:51.948102 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:51.951049 (+ 2947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:51.951062 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:51.951438 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:51.952565 (+ 1127us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:51.952630 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:51.957096 (+ 4466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:51.962201 (+ 5105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:51.962220 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:51.962246 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:51.963130 (+ 884us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:51.963653 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:51.963658 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:51.963660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:51.964068 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:51.964962 (+ 894us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:51.964971 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:51.965522 (+ 551us) client_negotiation.cc:769] Sending connection context
0811 00:25:51.965974 (+ 452us) client_negotiation.cc:240] Negotiation successful
0811 00:25:51.966793 (+ 819us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"thread_start_us":511,"threads_started":1}
I20260811 00:25:51.984012 4052 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:36815
I20260811 00:25:51.984091 4206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:36815 every 8 connection(s)
I20260811 00:25:51.985976 4052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:25:51.989459 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 4052
I20260811 00:25:51.989629 4207 heartbeater.cc:347] Connected to a master server at 127.2.120.62:43815
I20260811 00:25:51.989711 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:25:51.989921 4207 heartbeater.cc:464] Registering TS with master...
I20260811 00:25:51.990335 2528 external_mini_cluster.cc:1428] Setting key da02e38e6fb4789b752ea3e11664a605
I20260811 00:25:51.990963 4207 heartbeater.cc:511] Master 127.2.120.62:43815 requested a full tablet report, sending...
I20260811 00:25:51.992718 3815 ts_manager.cc:194] Registered new tserver with Master: 95af07ebb47248a6af9fa492eef17972 (127.2.120.2:36815)
I20260811 00:25:51.993690 4078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:51.977078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:41417 (local address 127.2.120.62:43815)
0811 00:25:51.977315 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:51.977325 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:51.977878 (+ 553us) server_negotiation.cc:407] Connection header received
0811 00:25:51.978055 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:51.978063 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:51.978181 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:51.978343 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:51.979427 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:51.980864 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:51.982403 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:51.983216 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:51.983389 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:51.986358 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:51.986366 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:51.986372 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:51.986419 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:51.990227 (+ 3808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:51.991013 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:51.991020 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:51.991024 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:51.991162 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:51.991553 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:51.991558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:51.991561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:51.992003 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:51.992170 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:51.992990 (+ 820us) server_negotiation.cc:299] Negotiation successful
0811 00:25:51.993292 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 00:25:51.993784 4202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:51.976996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:43815 (local address 127.2.120.2:41417)
0811 00:25:51.977747 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:51.977769 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:51.977916 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:51.978359 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:51.978363 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:51.978940 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:51.979250 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:51.979260 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:51.981059 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:51.981066 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:51.982227 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:51.982244 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:51.983781 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:51.983791 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:51.984019 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:51.984878 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:51.984908 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:51.986088 (+ 1180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:51.990380 (+ 4292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:51.990386 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:51.990391 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:51.990863 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:51.991282 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:51.991288 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:51.991291 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:51.991433 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:51.992365 (+ 932us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:51.992371 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:51.992674 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 00:25:51.992867 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 00:25:51.993263 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":413,"threads_started":1}
I20260811 00:25:51.993978 3815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:41981
WARNING: no policy specified for kudu/127.2.120.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
I20260811 00:25:52.041652 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43815
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:43911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:52.386627 4211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:52.386934 4211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:52.387037 4211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:52.394570 4211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:52.394698 4211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:52.394783 4211 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:25:52.394866 4211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:25:52.405174 4211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:43911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.120.62:43815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:52.406582 4211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:52.407985 4211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:52.422623 4218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:52.422669 4221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:52.423359 4217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:52.423902 4211 server_base.cc:1034] running on GCE node
I20260811 00:25:52.424803 4211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:52.426095 4211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:52.427381 4211 hybrid_clock.cc:648] HybridClock initialized: now 1786407952427315 us; error 40 us; skew 500 ppm
I20260811 00:25:52.427742 4211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:52 dist-test-slave-fd7l krb5kdc[3764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407952, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:52.431738 4211 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:25:52.433529 4211 webserver.cc:460] Webserver started at http://127.2.120.3:39721/ using document root <none> and password file <none>
I20260811 00:25:52.434358 4211 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:52.434512 4211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:52.434976 4211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:52.436105 4047 heartbeater.cc:503] Master 127.2.120.62:43815 was elected leader, sending a full tablet report...
I20260811 00:25:52.438966 4211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "9e87f62df5944bc199f0dac82a71e9fb"
format_stamp: "Formatted at 2026-08-11 00:25:52 on dist-test-slave-fd7l"
server_key: "6b6ae84f5ca477dc2eaafc1d1e5929a3"
server_key_iv: "ff631b0dc921ba119977c06ef886c5c6"
server_key_version: "encryptionkey@0"
I20260811 00:25:52.439960 4211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "9e87f62df5944bc199f0dac82a71e9fb"
format_stamp: "Formatted at 2026-08-11 00:25:52 on dist-test-slave-fd7l"
server_key: "6b6ae84f5ca477dc2eaafc1d1e5929a3"
server_key_iv: "ff631b0dc921ba119977c06ef886c5c6"
server_key_version: "encryptionkey@0"
I20260811 00:25:52.446489 4211 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 00:25:52.450436 4229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:52.451970 4211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:25:52.452139 4211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "9e87f62df5944bc199f0dac82a71e9fb"
format_stamp: "Formatted at 2026-08-11 00:25:52 on dist-test-slave-fd7l"
server_key: "6b6ae84f5ca477dc2eaafc1d1e5929a3"
server_key_iv: "ff631b0dc921ba119977c06ef886c5c6"
server_key_version: "encryptionkey@0"
I20260811 00:25:52.452374 4211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:52.472818 4211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:52.477336 4211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:52.477593 4211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:52.480419 4211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:25:52.480587 4211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:52.480762 4211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:25:52.480855 4211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:25:52 dist-test-slave-fd7l krb5kdc[3764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407952, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:25:52.525023 4078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:52.481202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:57993 (local address 127.2.120.62:43815)
0811 00:25:52.482260 (+ 1058us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:52.482268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:52.482865 (+ 597us) server_negotiation.cc:407] Connection header received
0811 00:25:52.485336 (+ 2471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:52.485342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:52.485460 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:52.485563 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:52.488156 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.489919 (+ 1763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.494938 (+ 5019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.496104 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.496554 (+ 450us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:52.507160 (+ 10606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:52.507181 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:52.507194 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:52.507342 (+ 148us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:52.514819 (+ 7477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:52.517803 (+ 2984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:52.517813 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:52.517817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:52.518203 (+ 386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:52.519392 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:52.519399 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:52.519402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:52.520084 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:52.520413 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:52.523136 (+ 2723us) server_negotiation.cc:299] Negotiation successful
0811 00:25:52.523922 (+ 786us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":864}
I20260811 00:25:52.527558 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:52.481701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:43815 (local address 127.2.120.3:57993)
0811 00:25:52.482659 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:52.482713 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:52.485016 (+ 2303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:52.486107 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:52.486126 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:52.486998 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:52.487897 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:52.487935 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.490542 (+ 2607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.490558 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:52.494191 (+ 3633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:52.494258 (+ 67us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.496771 (+ 2513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.496783 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:52.497281 (+ 498us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:52.499187 (+ 1906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:52.499262 (+ 75us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:52.506274 (+ 7012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:52.515430 (+ 9156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:52.515458 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:52.515508 (+ 50us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:52.517351 (+ 1843us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:52.518522 (+ 1171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:52.518529 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:52.518533 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:52.519063 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:52.520417 (+ 1354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:52.520441 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:52.521829 (+ 1388us) client_negotiation.cc:769] Sending connection context
0811 00:25:52.522665 (+ 836us) client_negotiation.cc:240] Negotiation successful
0811 00:25:52.523976 (+ 1311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":501,"threads_started":1}
I20260811 00:25:52.553895 4211 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:41655
I20260811 00:25:52.553915 4365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:41655 every 8 connection(s)
I20260811 00:25:52.555442 4078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:52.535450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:53955 (local address 127.2.120.62:43815)
0811 00:25:52.535675 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:52.535683 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:52.537110 (+ 1427us) server_negotiation.cc:407] Connection header received
0811 00:25:52.537331 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:52.537336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:52.537419 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:52.537580 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:52.539355 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.540677 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.543003 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.543529 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.543702 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:52.546926 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:52.546932 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:52.546935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:52.546972 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:52.550670 (+ 3698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:52.552188 (+ 1518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:52.552196 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:52.552200 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:52.552744 (+ 544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:52.553526 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:52.553535 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:52.553539 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:52.553944 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:52.554141 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:52.554808 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 00:25:52.555085 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 00:25:52.555527 4326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:52.535394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:43815 (local address 127.2.120.3:53955)
0811 00:25:52.536915 (+ 1521us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:52.536953 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:52.537181 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:52.537630 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:52.537638 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:52.538566 (+ 928us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:52.539142 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:52.539162 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.540848 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.540856 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:52.542812 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:52.542831 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.543754 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.543761 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:52.544003 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:52.544991 (+ 988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:52.545025 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:52.546662 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:52.550911 (+ 4249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:52.550917 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:52.550922 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:52.552030 (+ 1108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:52.553098 (+ 1068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:52.553108 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:52.553120 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:52.553339 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:52.554238 (+ 899us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:52.554242 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:52.554468 (+ 226us) client_negotiation.cc:769] Sending connection context
0811 00:25:52.554633 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 00:25:52.555085 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1296,"thread_start_us":1001,"threads_started":1}
I20260811 00:25:52.555613 4211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:25:52.557986 4366 heartbeater.cc:347] Connected to a master server at 127.2.120.62:43815
I20260811 00:25:52.558377 4366 heartbeater.cc:464] Registering TS with master...
I20260811 00:25:52.559289 4366 heartbeater.cc:511] Master 127.2.120.62:43815 requested a full tablet report, sending...
I20260811 00:25:52.560962 3815 ts_manager.cc:194] Registered new tserver with Master: 9e87f62df5944bc199f0dac82a71e9fb (127.2.120.3:41655)
I20260811 00:25:52.561908 3815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:57993
I20260811 00:25:52.565276 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 4211
I20260811 00:25:52.565447 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:25:52.565871 2528 external_mini_cluster.cc:1428] Setting key 4140c265768e5df60480d63734730389
I20260811 00:25:52.569859 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 00:25:52 dist-test-slave-fd7l krb5kdc[3764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407952, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 00:25:52 dist-test-slave-fd7l krb5kdc[3764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407952, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:25:52.625353 4078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:52.609784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41396 (local address 127.2.120.62:43815)
0811 00:25:52.610064 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:52.610072 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:52.610371 (+ 299us) server_negotiation.cc:407] Connection header received
0811 00:25:52.610641 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:52.610645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:52.610713 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:52.610824 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:52.612243 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.613069 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.614449 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.614852 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.615000 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:52.619105 (+ 4105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:52.619111 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:52.619114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:52.619153 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:52.622234 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:52.623067 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:52.623079 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:52.623082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:52.623264 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:52.623705 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:52.623710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:52.623712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:52.624046 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:52.624274 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:52.624750 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 00:25:52.625003 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 00:25:52.630168 3815 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:25:52.637420 3815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:25:52.663945 4385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:52.651884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41132 (local address 127.2.120.2:36815)
0811 00:25:52.655707 (+ 3823us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:52.655718 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:52.655754 (+ 36us) server_negotiation.cc:407] Connection header received
0811 00:25:52.655869 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:52.655881 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:52.656213 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:52.656532 (+ 319us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:52.657230 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.658599 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.661310 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.662585 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.663034 (+ 449us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:52.663310 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:52.663424 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 00:25:52.663611 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3543,"thread_start_us":3373,"threads_started":1}
I20260811 00:25:52.663996 4386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:52.651350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:36815 (local address 127.0.0.1:41132)
0811 00:25:52.655223 (+ 3873us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:52.655243 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:52.655363 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:52.656463 (+ 1100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:52.656468 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:52.656501 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:52.657010 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:52.657023 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.658799 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.658807 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:52.661119 (+ 2312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:52.661134 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.662819 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.662827 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:52.663071 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:52.663136 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 00:25:52.663270 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 00:25:52.663501 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3719,"mutex_wait_us":23,"thread_start_us":4295,"threads_started":1}
I20260811 00:25:52.665568 4382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:52.651062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:39645 (local address 127.0.0.1:53356)
0811 00:25:52.653635 (+ 2573us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:52.653674 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:52.653964 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:52.655101 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:52.655115 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:52.655146 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:52.655683 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:52.655705 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.657835 (+ 2130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.657841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:52.661838 (+ 3997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:52.661856 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.664621 (+ 2765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.664629 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:52.664855 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:52.664897 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 00:25:52.665010 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 00:25:52.665158 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2301,"mutex_wait_us":649,"thread_start_us":2155,"threads_started":1}
I20260811 00:25:52.665576 4383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:52.651251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53356 (local address 127.2.120.1:39645)
0811 00:25:52.653038 (+ 1787us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:52.653053 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:52.653799 (+ 746us) server_negotiation.cc:407] Connection header received
0811 00:25:52.654244 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:52.654255 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:52.654555 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:52.654735 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:52.655906 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.657692 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.662069 (+ 4377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.663631 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.663771 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:52.663976 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:52.665080 (+ 1104us) server_negotiation.cc:299] Negotiation successful
0811 00:25:52.665288 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1554,"thread_start_us":2688,"threads_started":1}
I20260811 00:25:52.668948 4127 tablet_service.cc:1467] Processing CreateTablet for tablet f5100b24bf5f495bbff398f830f6d5f2 (DEFAULT_TABLE table=test-table [id=22d12828a0c94c249ea51ef26d9bfa77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:25:52.669481 3968 tablet_service.cc:1467] Processing CreateTablet for tablet f5100b24bf5f495bbff398f830f6d5f2 (DEFAULT_TABLE table=test-table [id=22d12828a0c94c249ea51ef26d9bfa77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:25:52.670576 4127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5100b24bf5f495bbff398f830f6d5f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:52.670684 3968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5100b24bf5f495bbff398f830f6d5f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:52.677980 4384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:52.652737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:41655 (local address 127.0.0.1:47550)
0811 00:25:52.655927 (+ 3190us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:52.655946 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:52.656158 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:52.657037 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:52.657045 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:52.657110 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:52.657583 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:52.657594 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.659936 (+ 2342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.659942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:52.662373 (+ 2431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:52.662386 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.677181 (+ 14795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.677188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:52.677364 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:52.677394 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 00:25:52.677508 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 00:25:52.677638 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3046,"mutex_wait_us":26,"thread_start_us":2142,"threads_started":1}
I20260811 00:25:52.678208 4387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:52.653544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47550 (local address 127.2.120.3:41655)
0811 00:25:52.654707 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:52.654715 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:52.656048 (+ 1333us) server_negotiation.cc:407] Connection header received
0811 00:25:52.656348 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:52.656358 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:52.656618 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:52.656774 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:52.657865 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.659728 (+ 1863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.662576 (+ 2848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:52.665679 (+ 3103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:52.676736 (+ 11057us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:52.677008 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:52.677615 (+ 607us) server_negotiation.cc:299] Negotiation successful
0811 00:25:52.677852 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":971,"thread_start_us":755,"threads_started":1}
I20260811 00:25:52.683045 4284 tablet_service.cc:1467] Processing CreateTablet for tablet f5100b24bf5f495bbff398f830f6d5f2 (DEFAULT_TABLE table=test-table [id=22d12828a0c94c249ea51ef26d9bfa77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:25:52.684064 4388 tablet_bootstrap.cc:492] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Bootstrap starting.
I20260811 00:25:52.686102 4284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5100b24bf5f495bbff398f830f6d5f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:52.688393 4388 tablet_bootstrap.cc:654] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:52.688603 4389 tablet_bootstrap.cc:492] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972: Bootstrap starting.
I20260811 00:25:52.689973 4388 log.cc:824] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:52.692557 4389 tablet_bootstrap.cc:654] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:52.693965 4388 tablet_bootstrap.cc:492] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0: No bootstrap required, opened a new log
I20260811 00:25:52.694147 4389 log.cc:824] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:52.694370 4388 ts_tablet_manager.cc:1397] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 00:25:52.698112 4389 tablet_bootstrap.cc:492] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972: No bootstrap required, opened a new log
I20260811 00:25:52.698513 4389 ts_tablet_manager.cc:1397] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 00:25:52.706179 4392 tablet_bootstrap.cc:492] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb: Bootstrap starting.
I20260811 00:25:52.706426 4388 raft_consensus.cc:348] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:52.706948 4388 raft_consensus.cc:374] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:52.707080 4388 raft_consensus.cc:729] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba1bd2b484e4d648a76ef97a1fcdbb0, State: Initialized, Role: FOLLOWER
I20260811 00:25:52.707837 4388 consensus_queue.cc:260] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:52.709946 4388 ts_tablet_manager.cc:1428] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260811 00:25:52.710768 4389 raft_consensus.cc:348] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:52.711292 4389 raft_consensus.cc:374] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:52.711448 4389 raft_consensus.cc:729] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95af07ebb47248a6af9fa492eef17972, State: Initialized, Role: FOLLOWER
I20260811 00:25:52.711444 4392 tablet_bootstrap.cc:654] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:52.712144 4389 consensus_queue.cc:260] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:52.713006 4392 log.cc:824] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:52.713518 4207 heartbeater.cc:503] Master 127.2.120.62:43815 was elected leader, sending a full tablet report...
I20260811 00:25:52.714223 4389 ts_tablet_manager.cc:1428] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260811 00:25:52.718299 4392 tablet_bootstrap.cc:492] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb: No bootstrap required, opened a new log
I20260811 00:25:52.718652 4392 ts_tablet_manager.cc:1397] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.004s
I20260811 00:25:52.727345 4392 raft_consensus.cc:348] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:52.727869 4392 raft_consensus.cc:374] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:52.728015 4392 raft_consensus.cc:729] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e87f62df5944bc199f0dac82a71e9fb, State: Initialized, Role: FOLLOWER
I20260811 00:25:52.728866 4392 consensus_queue.cc:260] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:52.730252 4366 heartbeater.cc:503] Master 127.2.120.62:43815 was elected leader, sending a full tablet report...
I20260811 00:25:52.730962 4392 ts_tablet_manager.cc:1428] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb: Time spent starting tablet: real 0.012s user 0.010s sys 0.003s
W20260811 00:25:52.739499 4208 tablet.cc:2369] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:25:52.807878 4367 tablet.cc:2369] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:25:52.927623 4049 tablet.cc:2369] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:25:53.003310 4397 raft_consensus.cc:482] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:25:53.003659 4397 raft_consensus.cc:504] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.005699 4397 leader_election.cc:288] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ba1bd2b484e4d648a76ef97a1fcdbb0 (127.2.120.1:39645), 9e87f62df5944bc199f0dac82a71e9fb (127.2.120.3:41655)
I20260811 00:25:53.013742 4383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.006319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:57175 (local address 127.2.120.1:39645)
0811 00:25:53.006574 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:53.006581 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:53.007310 (+ 729us) server_negotiation.cc:407] Connection header received
0811 00:25:53.007583 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:53.007588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:53.007666 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:53.007771 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:53.008726 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.009693 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.011793 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.012675 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.012836 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.012954 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:53.013317 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 00:25:53.013457 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 00:25:53.013741 4387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.006369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:34303 (local address 127.2.120.3:41655)
0811 00:25:53.006640 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:53.006649 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:53.007481 (+ 832us) server_negotiation.cc:407] Connection header received
0811 00:25:53.007700 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:53.007704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:53.007831 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:53.007938 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:53.008831 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.009637 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.011776 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.012648 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.012804 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.012930 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:53.013317 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 00:25:53.013458 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260811 00:25:53.013895 4401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.006182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:41655 (local address 127.2.120.2:34303)
0811 00:25:53.007358 (+ 1176us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:53.007383 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:53.007551 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:53.007985 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:53.007991 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:53.008055 (+ 64us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:53.008672 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:53.008683 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.009775 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.009783 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:53.011604 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:53.011618 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.012884 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.012890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:53.013119 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.013171 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 00:25:53.013285 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 00:25:53.013438 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1024,"mutex_wait_us":53,"thread_start_us":826,"threads_started":1}
I20260811 00:25:53.014375 4400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.005973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:39645 (local address 127.2.120.2:57175)
0811 00:25:53.007145 (+ 1172us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:53.007175 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:53.007419 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:53.007789 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:53.007793 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:53.007819 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:53.008496 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:53.008559 (+ 63us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.009838 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.009845 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:53.011623 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:53.011637 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.012926 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.012934 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:53.013124 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.013181 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 00:25:53.013286 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 00:25:53.013438 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"mutex_wait_us":29,"thread_start_us":632,"threads_started":1}
I20260811 00:25:53.014950 4308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5100b24bf5f495bbff398f830f6d5f2" candidate_uuid: "95af07ebb47248a6af9fa492eef17972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e87f62df5944bc199f0dac82a71e9fb" is_pre_election: true
I20260811 00:25:53.015563 4308 raft_consensus.cc:2454] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95af07ebb47248a6af9fa492eef17972 in term 0.
I20260811 00:25:53.015933 3997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5100b24bf5f495bbff398f830f6d5f2" candidate_uuid: "95af07ebb47248a6af9fa492eef17972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" is_pre_election: true
I20260811 00:25:53.016271 4070 leader_election.cc:302] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95af07ebb47248a6af9fa492eef17972, 9e87f62df5944bc199f0dac82a71e9fb; no voters:
I20260811 00:25:53.016407 3997 raft_consensus.cc:2454] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95af07ebb47248a6af9fa492eef17972 in term 0.
I20260811 00:25:53.016736 4397 raft_consensus.cc:2781] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:25:53.016897 4397 raft_consensus.cc:482] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:25:53.017022 4397 raft_consensus.cc:3037] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:53.019289 4397 raft_consensus.cc:504] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.019936 4397 leader_election.cc:288] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [CANDIDATE]: Term 1 election: Requested vote from peers 7ba1bd2b484e4d648a76ef97a1fcdbb0 (127.2.120.1:39645), 9e87f62df5944bc199f0dac82a71e9fb (127.2.120.3:41655)
I20260811 00:25:53.020569 3997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5100b24bf5f495bbff398f830f6d5f2" candidate_uuid: "95af07ebb47248a6af9fa492eef17972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0"
I20260811 00:25:53.020774 4308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5100b24bf5f495bbff398f830f6d5f2" candidate_uuid: "95af07ebb47248a6af9fa492eef17972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e87f62df5944bc199f0dac82a71e9fb"
I20260811 00:25:53.020817 3997 raft_consensus.cc:3037] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:53.021095 4308 raft_consensus.cc:3037] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:53.023517 4308 raft_consensus.cc:2454] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95af07ebb47248a6af9fa492eef17972 in term 1.
I20260811 00:25:53.023612 3997 raft_consensus.cc:2454] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95af07ebb47248a6af9fa492eef17972 in term 1.
I20260811 00:25:53.024410 4070 leader_election.cc:302] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95af07ebb47248a6af9fa492eef17972, 9e87f62df5944bc199f0dac82a71e9fb; no voters:
I20260811 00:25:53.024792 4397 raft_consensus.cc:2781] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:53.026181 4397 raft_consensus.cc:686] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [term 1 LEADER]: Becoming Leader. State: Replica: 95af07ebb47248a6af9fa492eef17972, State: Running, Role: LEADER
I20260811 00:25:53.026968 4397 consensus_queue.cc:237] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.032394 3814 catalog_manager.cc:5495] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95af07ebb47248a6af9fa492eef17972 (127.2.120.2). New cstate: current_term: 1 leader_uuid: "95af07ebb47248a6af9fa492eef17972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } health_report { overall_health: UNKNOWN } } }
I20260811 00:25:53.142367 4385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.136054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41148 (local address 127.2.120.2:36815)
0811 00:25:53.136316 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:53.136323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:53.136648 (+ 325us) server_negotiation.cc:407] Connection header received
0811 00:25:53.136949 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:53.136954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:53.137042 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:53.137185 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:53.137650 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.138363 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.139734 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.140218 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.140366 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.140968 (+ 602us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:53.141664 (+ 696us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:53.141785 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:53.141954 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 00:25:53.142034 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 00:25:53.153208 3997 raft_consensus.cc:1261] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 1 FOLLOWER]: Refusing update from remote peer 95af07ebb47248a6af9fa492eef17972: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:25:53.153208 4308 raft_consensus.cc:1261] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb [term 1 FOLLOWER]: Refusing update from remote peer 95af07ebb47248a6af9fa492eef17972: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:25:53.154737 4402 consensus_queue.cc:1035] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:25:53.155159 4397 consensus_queue.cc:1035] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:25:53.169196 4406 mvcc.cc:205] Tried to move back new op lower bound from 7317126976101355520 to 7317126975602614272. Current Snapshot: MvccSnapshot[applied={T|T < 7317126976101355520}]
I20260811 00:25:53.170538 4408 mvcc.cc:205] Tried to move back new op lower bound from 7317126976101355520 to 7317126975602614272. Current Snapshot: MvccSnapshot[applied={T|T < 7317126976101355520}]
I20260811 00:25:53.195135 3814 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:25:53.195487 3814 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:25:53.200083 3814 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858: Sending DeleteTablet for 3 replicas of tablet f5100b24bf5f495bbff398f830f6d5f2
I20260811 00:25:53.201653 4127 tablet_service.cc:1514] Processing DeleteTablet for tablet f5100b24bf5f495bbff398f830f6d5f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:25:53 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:41132
I20260811 00:25:53.201624 3968 tablet_service.cc:1514] Processing DeleteTablet for tablet f5100b24bf5f495bbff398f830f6d5f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:25:53 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:53356
I20260811 00:25:53.202216 4284 tablet_service.cc:1514] Processing DeleteTablet for tablet f5100b24bf5f495bbff398f830f6d5f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:25:53 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:47550
I20260811 00:25:53.206410 3814 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:25:53.209688 3814 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:25:53.213665 4416 tablet_replica.cc:330] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0: stopping tablet replica
I20260811 00:25:53.214318 4418 tablet_replica.cc:330] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb: stopping tablet replica
I20260811 00:25:53.214890 4416 raft_consensus.cc:2229] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:25:53.216028 4416 raft_consensus.cc:2258] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:25:53.217041 4418 raft_consensus.cc:2229] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:25:53.218767 4417 tablet_replica.cc:330] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972: stopping tablet replica
I20260811 00:25:53.219904 4417 raft_consensus.cc:2229] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:25:53.220207 4418 raft_consensus.cc:2258] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:25:53.223183 4417 raft_consensus.cc:2258] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:25:53.225733 4418 ts_tablet_manager.cc:1905] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:25:53.225802 4416 ts_tablet_manager.cc:1905] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:25:53.235483 4417 ts_tablet_manager.cc:1905] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:25:53.235131 3968 tablet_service.cc:1467] Processing CreateTablet for tablet b14e3be81c8c419e99104b0aa094345a (DEFAULT_TABLE table=test-table [id=a55e4c84969944fd9c6b7ad5a6360dff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:25:53.236423 3968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b14e3be81c8c419e99104b0aa094345a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:53.240640 4418 ts_tablet_manager.cc:1918] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:25:53.240907 4418 log.cc:1196] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/f5100b24bf5f495bbff398f830f6d5f2
I20260811 00:25:53.241101 4416 ts_tablet_manager.cc:1918] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:25:53.241305 4416 log.cc:1196] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/f5100b24bf5f495bbff398f830f6d5f2
I20260811 00:25:53.241549 4418 ts_tablet_manager.cc:1939] T f5100b24bf5f495bbff398f830f6d5f2 P 9e87f62df5944bc199f0dac82a71e9fb: Deleting consensus metadata
I20260811 00:25:53.243084 4127 tablet_service.cc:1467] Processing CreateTablet for tablet b14e3be81c8c419e99104b0aa094345a (DEFAULT_TABLE table=test-table [id=a55e4c84969944fd9c6b7ad5a6360dff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:25:53.243677 4127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b14e3be81c8c419e99104b0aa094345a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:53.244155 4416 ts_tablet_manager.cc:1939] T f5100b24bf5f495bbff398f830f6d5f2 P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Deleting consensus metadata
I20260811 00:25:53.245191 3795 catalog_manager.cc:4841] TS 9e87f62df5944bc199f0dac82a71e9fb (127.2.120.3:41655): tablet f5100b24bf5f495bbff398f830f6d5f2 (table test-table [id=22d12828a0c94c249ea51ef26d9bfa77]) successfully deleted
I20260811 00:25:53.245723 4284 tablet_service.cc:1467] Processing CreateTablet for tablet b14e3be81c8c419e99104b0aa094345a (DEFAULT_TABLE table=test-table [id=a55e4c84969944fd9c6b7ad5a6360dff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:25:53.246244 4284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b14e3be81c8c419e99104b0aa094345a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:53.246363 3799 catalog_manager.cc:4841] TS 7ba1bd2b484e4d648a76ef97a1fcdbb0 (127.2.120.1:39645): tablet f5100b24bf5f495bbff398f830f6d5f2 (table test-table [id=22d12828a0c94c249ea51ef26d9bfa77]) successfully deleted
I20260811 00:25:53.259004 4419 tablet_bootstrap.cc:492] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Bootstrap starting.
I20260811 00:25:53.260658 4417 ts_tablet_manager.cc:1918] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:25:53.260912 4417 log.cc:1196] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/f5100b24bf5f495bbff398f830f6d5f2
I20260811 00:25:53.261391 4419 tablet_bootstrap.cc:654] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:53.261615 4417 ts_tablet_manager.cc:1939] T f5100b24bf5f495bbff398f830f6d5f2 P 95af07ebb47248a6af9fa492eef17972: Deleting consensus metadata
I20260811 00:25:53.264058 4421 tablet_bootstrap.cc:492] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb: Bootstrap starting.
I20260811 00:25:53.264191 3801 catalog_manager.cc:4841] TS 95af07ebb47248a6af9fa492eef17972 (127.2.120.2:36815): tablet f5100b24bf5f495bbff398f830f6d5f2 (table test-table [id=22d12828a0c94c249ea51ef26d9bfa77]) successfully deleted
I20260811 00:25:53.266232 4420 tablet_bootstrap.cc:492] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972: Bootstrap starting.
I20260811 00:25:53.266327 4421 tablet_bootstrap.cc:654] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:53.268478 4419 tablet_bootstrap.cc:492] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0: No bootstrap required, opened a new log
I20260811 00:25:53.268761 4419 ts_tablet_manager.cc:1397] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.001s
I20260811 00:25:53.269754 4419 raft_consensus.cc:348] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.270072 4419 raft_consensus.cc:374] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:53.270188 4419 raft_consensus.cc:729] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba1bd2b484e4d648a76ef97a1fcdbb0, State: Initialized, Role: FOLLOWER
I20260811 00:25:53.270648 4420 tablet_bootstrap.cc:654] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:53.270519 4419 consensus_queue.cc:260] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.272403 4419 ts_tablet_manager.cc:1428] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260811 00:25:53.274755 4421 tablet_bootstrap.cc:492] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb: No bootstrap required, opened a new log
I20260811 00:25:53.274974 4421 ts_tablet_manager.cc:1397] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260811 00:25:53.275934 4421 raft_consensus.cc:348] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.276240 4421 raft_consensus.cc:374] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:53.276360 4421 raft_consensus.cc:729] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e87f62df5944bc199f0dac82a71e9fb, State: Initialized, Role: FOLLOWER
I20260811 00:25:53.276680 4421 consensus_queue.cc:260] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.277470 4421 ts_tablet_manager.cc:1428] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 00:25:53.280129 4420 tablet_bootstrap.cc:492] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972: No bootstrap required, opened a new log
I20260811 00:25:53.280339 4420 ts_tablet_manager.cc:1397] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.008s
I20260811 00:25:53.282455 4420 raft_consensus.cc:348] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.282711 4420 raft_consensus.cc:374] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:53.282837 4420 raft_consensus.cc:729] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95af07ebb47248a6af9fa492eef17972, State: Initialized, Role: FOLLOWER
I20260811 00:25:53.283129 4420 consensus_queue.cc:260] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.283722 4420 ts_tablet_manager.cc:1428] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 00:25:53.304556 4411 raft_consensus.cc:482] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:25:53.304939 4411 raft_consensus.cc:504] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.306027 4411 leader_election.cc:288] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ba1bd2b484e4d648a76ef97a1fcdbb0 (127.2.120.1:39645), 9e87f62df5944bc199f0dac82a71e9fb (127.2.120.3:41655)
I20260811 00:25:53.306669 4308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b14e3be81c8c419e99104b0aa094345a" candidate_uuid: "95af07ebb47248a6af9fa492eef17972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e87f62df5944bc199f0dac82a71e9fb" is_pre_election: true
I20260811 00:25:53.306730 3997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b14e3be81c8c419e99104b0aa094345a" candidate_uuid: "95af07ebb47248a6af9fa492eef17972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" is_pre_election: true
I20260811 00:25:53.306958 4308 raft_consensus.cc:2454] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95af07ebb47248a6af9fa492eef17972 in term 0.
I20260811 00:25:53.307011 3997 raft_consensus.cc:2454] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95af07ebb47248a6af9fa492eef17972 in term 0.
I20260811 00:25:53.307505 4072 leader_election.cc:302] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7ba1bd2b484e4d648a76ef97a1fcdbb0, 95af07ebb47248a6af9fa492eef17972, 9e87f62df5944bc199f0dac82a71e9fb; no voters:
I20260811 00:25:53.307940 4411 raft_consensus.cc:2781] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:25:53.308090 4411 raft_consensus.cc:482] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:25:53.308202 4411 raft_consensus.cc:3037] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:53.312134 4411 raft_consensus.cc:504] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.312937 4411 leader_election.cc:288] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [CANDIDATE]: Term 1 election: Requested vote from peers 7ba1bd2b484e4d648a76ef97a1fcdbb0 (127.2.120.1:39645), 9e87f62df5944bc199f0dac82a71e9fb (127.2.120.3:41655)
I20260811 00:25:53.313508 3997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b14e3be81c8c419e99104b0aa094345a" candidate_uuid: "95af07ebb47248a6af9fa492eef17972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0"
I20260811 00:25:53.313547 4308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b14e3be81c8c419e99104b0aa094345a" candidate_uuid: "95af07ebb47248a6af9fa492eef17972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e87f62df5944bc199f0dac82a71e9fb"
I20260811 00:25:53.313745 3997 raft_consensus.cc:3037] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:53.313802 4308 raft_consensus.cc:3037] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:53.315894 3997 raft_consensus.cc:2454] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95af07ebb47248a6af9fa492eef17972 in term 1.
I20260811 00:25:53.316411 4072 leader_election.cc:302] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ba1bd2b484e4d648a76ef97a1fcdbb0, 95af07ebb47248a6af9fa492eef17972; no voters:
I20260811 00:25:53.316805 4411 raft_consensus.cc:2781] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:53.317040 4411 raft_consensus.cc:686] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [term 1 LEADER]: Becoming Leader. State: Replica: 95af07ebb47248a6af9fa492eef17972, State: Running, Role: LEADER
I20260811 00:25:53.317088 4308 raft_consensus.cc:2454] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95af07ebb47248a6af9fa492eef17972 in term 1.
I20260811 00:25:53.317353 4411 consensus_queue.cc:237] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.320432 3816 catalog_manager.cc:5495] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95af07ebb47248a6af9fa492eef17972 (127.2.120.2). New cstate: current_term: 1 leader_uuid: "95af07ebb47248a6af9fa492eef17972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } health_report { overall_health: UNKNOWN } } }
Aug 11 00:25:53 dist-test-slave-fd7l krb5kdc[3764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407950, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:25:53.391811 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.371900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:43815 (local address 127.0.0.1:47320)
0811 00:25:53.373806 (+ 1906us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:53.373829 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:53.374045 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:53.374501 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:53.374506 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:53.375440 (+ 934us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:53.375786 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:53.375801 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.377247 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.377253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:53.378849 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:53.378866 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.379731 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.379738 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:53.380024 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.380936 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:53.380961 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:53.384313 (+ 3352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:53.389031 (+ 4718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:53.389049 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:53.389057 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:53.389602 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:53.390065 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:53.390069 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:53.390071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:53.390259 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:53.390914 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:53.390922 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:53.391093 (+ 171us) client_negotiation.cc:769] Sending connection context
0811 00:25:53.391247 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 00:25:53.391486 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1750,"thread_start_us":1458,"threads_started":1}
I20260811 00:25:53.392150 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.371901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47320 (local address 127.2.120.62:43815)
0811 00:25:53.373178 (+ 1277us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:53.373187 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:53.374044 (+ 857us) server_negotiation.cc:407] Connection header received
0811 00:25:53.374232 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:53.374236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:53.374301 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:53.374419 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:53.375991 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.377090 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.379015 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.379538 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.379649 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.384653 (+ 5004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:53.384662 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:53.384668 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:53.384713 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:53.388841 (+ 4128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:53.389745 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:53.389752 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:53.389756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:53.389921 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:53.390378 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:53.390383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:53.390386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:53.390773 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:53.390932 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:53.391393 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 00:25:53.391655 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1030,"thread_start_us":812,"threads_started":1}
I20260811 00:25:53.398602 3816 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:47320:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 00:25:53.400031 3816 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:25:53.409593 4284 tablet_service.cc:1467] Processing CreateTablet for tablet 17468bce420c4b619565a36c6cdbfbfb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1bad277a18c84d1c87598f7877fa8285]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:25:53.409852 4127 tablet_service.cc:1467] Processing CreateTablet for tablet 17468bce420c4b619565a36c6cdbfbfb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1bad277a18c84d1c87598f7877fa8285]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:25:53.409963 3968 tablet_service.cc:1467] Processing CreateTablet for tablet 17468bce420c4b619565a36c6cdbfbfb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1bad277a18c84d1c87598f7877fa8285]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:25:53.410290 4284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17468bce420c4b619565a36c6cdbfbfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:53.410574 3968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17468bce420c4b619565a36c6cdbfbfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:53.410584 4127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17468bce420c4b619565a36c6cdbfbfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:53.419322 4420 tablet_bootstrap.cc:492] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972: Bootstrap starting.
I20260811 00:25:53.419366 4419 tablet_bootstrap.cc:492] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Bootstrap starting.
I20260811 00:25:53.421847 4419 tablet_bootstrap.cc:654] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:53.421918 4420 tablet_bootstrap.cc:654] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:53.433832 4420 tablet_bootstrap.cc:492] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972: No bootstrap required, opened a new log
I20260811 00:25:53.434095 4420 ts_tablet_manager.cc:1397] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260811 00:25:53.435111 4420 raft_consensus.cc:348] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.435395 4421 tablet_bootstrap.cc:492] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb: Bootstrap starting.
I20260811 00:25:53.435438 4420 raft_consensus.cc:374] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:53.435549 4420 raft_consensus.cc:729] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95af07ebb47248a6af9fa492eef17972, State: Initialized, Role: FOLLOWER
I20260811 00:25:53.435878 4420 consensus_queue.cc:260] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.437158 4419 tablet_bootstrap.cc:492] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0: No bootstrap required, opened a new log
I20260811 00:25:53.437371 4419 ts_tablet_manager.cc:1397] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.007s
I20260811 00:25:53.437460 4421 tablet_bootstrap.cc:654] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:53.438119 4419 raft_consensus.cc:348] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.438377 4419 raft_consensus.cc:374] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:53.438465 4419 raft_consensus.cc:729] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba1bd2b484e4d648a76ef97a1fcdbb0, State: Initialized, Role: FOLLOWER
I20260811 00:25:53.438830 4419 consensus_queue.cc:260] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.439491 4419 ts_tablet_manager.cc:1428] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 00:25:53.439383 4395 tablet_replica.cc:439] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } } }
I20260811 00:25:53.439806 4395 tablet_replica.cc:442] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:25:53.439920 4411 tablet_replica.cc:439] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } } }
I20260811 00:25:53.440255 4411 tablet_replica.cc:442] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:25:53.441588 4420 ts_tablet_manager.cc:1428] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972: Time spent starting tablet: real 0.007s user 0.001s sys 0.002s
I20260811 00:25:53.450501 4421 tablet_bootstrap.cc:492] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb: No bootstrap required, opened a new log
I20260811 00:25:53.450749 4421 ts_tablet_manager.cc:1397] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260811 00:25:53.451700 4421 raft_consensus.cc:348] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.452024 4421 raft_consensus.cc:374] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:53.452101 4421 raft_consensus.cc:729] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e87f62df5944bc199f0dac82a71e9fb, State: Initialized, Role: FOLLOWER
I20260811 00:25:53.452401 4421 consensus_queue.cc:260] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.452997 4399 tablet_replica.cc:439] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } } }
I20260811 00:25:53.453392 4399 tablet_replica.cc:442] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:25:53.454746 4421 ts_tablet_manager.cc:1428] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 00:25:53.607793 4399 raft_consensus.cc:482] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:25:53.608400 4399 raft_consensus.cc:504] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.613189 4399 leader_election.cc:288] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ba1bd2b484e4d648a76ef97a1fcdbb0 (127.2.120.1:39645), 95af07ebb47248a6af9fa492eef17972 (127.2.120.2:36815)
I20260811 00:25:53.635641 4385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.615856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:56933 (local address 127.2.120.2:36815)
0811 00:25:53.616365 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:53.616375 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:53.627028 (+ 10653us) server_negotiation.cc:407] Connection header received
0811 00:25:53.627517 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:53.627524 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:53.627629 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:53.627725 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:53.629019 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.630449 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.633295 (+ 2846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.634283 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.634505 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.634634 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:53.635076 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 00:25:53.635326 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221}
I20260811 00:25:53.635689 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.614572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:36815 (local address 127.2.120.3:56933)
0811 00:25:53.626806 (+ 12234us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:53.626846 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:53.627275 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:53.628146 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:53.628153 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:53.628190 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:53.628742 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:53.628766 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.630695 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.630704 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:53.633062 (+ 2358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:53.633079 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.634575 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.634582 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:53.634825 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.634866 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 00:25:53.634983 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 00:25:53.635147 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11966,"mutex_wait_us":27,"thread_start_us":10224,"threads_started":1}
I20260811 00:25:53.637089 4149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17468bce420c4b619565a36c6cdbfbfb" candidate_uuid: "9e87f62df5944bc199f0dac82a71e9fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95af07ebb47248a6af9fa492eef17972" is_pre_election: true
I20260811 00:25:53.637786 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.614345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:57477 (local address 127.2.120.1:39645)
0811 00:25:53.629419 (+ 15074us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:53.629425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:53.629450 (+ 25us) server_negotiation.cc:407] Connection header received
0811 00:25:53.629563 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:53.629567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:53.629639 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:53.629802 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:53.630614 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.632050 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.634828 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.636456 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.636774 (+ 318us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.636921 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:53.637266 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 00:25:53.637429 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14890,"thread_start_us":14671,"threads_started":1}
I20260811 00:25:53.637636 4149 raft_consensus.cc:2454] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e87f62df5944bc199f0dac82a71e9fb in term 0.
I20260811 00:25:53.637840 4441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.613788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:39645 (local address 127.2.120.3:57477)
0811 00:25:53.625904 (+ 12116us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:53.625980 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:53.626524 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:53.629824 (+ 3300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:53.629830 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:53.629861 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:53.630341 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:53.630354 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.632248 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.632254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:53.634589 (+ 2335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:53.634610 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.636833 (+ 2223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.636841 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:53.637048 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.637103 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 00:25:53.637232 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 00:25:53.637397 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11510,"mutex_wait_us":4,"thread_start_us":14681,"threads_started":1}
I20260811 00:25:53.638671 4235 leader_election.cc:302] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95af07ebb47248a6af9fa492eef17972, 9e87f62df5944bc199f0dac82a71e9fb; no voters:
I20260811 00:25:53.639153 4399 raft_consensus.cc:2781] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:25:53.639307 4399 raft_consensus.cc:482] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:25:53.639381 3997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17468bce420c4b619565a36c6cdbfbfb" candidate_uuid: "9e87f62df5944bc199f0dac82a71e9fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" is_pre_election: true
I20260811 00:25:53.639534 4399 raft_consensus.cc:3037] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:53.639739 3997 raft_consensus.cc:2454] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e87f62df5944bc199f0dac82a71e9fb in term 0.
I20260811 00:25:53.642603 4399 raft_consensus.cc:504] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.643553 4399 leader_election.cc:288] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [CANDIDATE]: Term 1 election: Requested vote from peers 7ba1bd2b484e4d648a76ef97a1fcdbb0 (127.2.120.1:39645), 95af07ebb47248a6af9fa492eef17972 (127.2.120.2:36815)
I20260811 00:25:53.644088 3997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17468bce420c4b619565a36c6cdbfbfb" candidate_uuid: "9e87f62df5944bc199f0dac82a71e9fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0"
I20260811 00:25:53.644213 4149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17468bce420c4b619565a36c6cdbfbfb" candidate_uuid: "9e87f62df5944bc199f0dac82a71e9fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95af07ebb47248a6af9fa492eef17972"
I20260811 00:25:53.644397 3997 raft_consensus.cc:3037] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:53.644469 4149 raft_consensus.cc:3037] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:53.647918 3997 raft_consensus.cc:2454] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e87f62df5944bc199f0dac82a71e9fb in term 1.
I20260811 00:25:53.648341 4149 raft_consensus.cc:2454] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e87f62df5944bc199f0dac82a71e9fb in term 1.
I20260811 00:25:53.648725 4234 leader_election.cc:302] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ba1bd2b484e4d648a76ef97a1fcdbb0, 9e87f62df5944bc199f0dac82a71e9fb; no voters:
I20260811 00:25:53.649132 4399 raft_consensus.cc:2781] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:53.649720 4399 raft_consensus.cc:686] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [term 1 LEADER]: Becoming Leader. State: Replica: 9e87f62df5944bc199f0dac82a71e9fb, State: Running, Role: LEADER
I20260811 00:25:53.650246 4399 consensus_queue.cc:237] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } }
I20260811 00:25:53.653057 4442 tablet_replica.cc:439] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb: TxnStatusTablet state changed. Reason: New leader 9e87f62df5944bc199f0dac82a71e9fb. Latest consensus state: current_term: 1 leader_uuid: "9e87f62df5944bc199f0dac82a71e9fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } } }
I20260811 00:25:53.653497 4442 tablet_replica.cc:442] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:25:53.654299 4444 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 00:25:53.654567 4444 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 00:25:53.654770 3814 catalog_manager.cc:5495] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e87f62df5944bc199f0dac82a71e9fb (127.2.120.3). New cstate: current_term: 1 leader_uuid: "9e87f62df5944bc199f0dac82a71e9fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } health_report { overall_health: HEALTHY } } }
I20260811 00:25:53.694257 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.684695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58802 (local address 127.2.120.3:41655)
0811 00:25:53.686108 (+ 1413us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:53.686115 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:53.686137 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:25:53.686246 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:53.686252 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:53.686372 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:53.686579 (+ 207us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:53.687063 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.688576 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.690863 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.691661 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.691819 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.692543 (+ 724us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:53.693426 (+ 883us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:53.693636 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:53.693723 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 00:25:53.693827 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1266,"thread_start_us":1120,"threads_started":1}
I20260811 00:25:53.694458 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.684751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:41655 (local address 127.0.0.1:58802)
0811 00:25:53.685191 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:53.685216 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:53.685433 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:53.686619 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:53.686624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:53.686638 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:25:53.686876 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:53.686892 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.688788 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.688797 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:53.690634 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:53.690662 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.691864 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.691871 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:53.692175 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.692269 (+ 94us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:25:53.693566 (+ 1297us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:25:53.693593 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 00:25:53.693683 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 00:25:53.694072 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237}
I20260811 00:25:53.700227 3997 raft_consensus.cc:1261] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 1 FOLLOWER]: Refusing update from remote peer 9e87f62df5944bc199f0dac82a71e9fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:25:53.700459 4149 raft_consensus.cc:1261] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972 [term 1 FOLLOWER]: Refusing update from remote peer 9e87f62df5944bc199f0dac82a71e9fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:25:53.701289 4442 consensus_queue.cc:1035] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:25:53.701726 4399 consensus_queue.cc:1035] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:25:53.722548 4411 tablet_replica.cc:439] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972: TxnStatusTablet state changed. Reason: New leader 9e87f62df5944bc199f0dac82a71e9fb. Latest consensus state: current_term: 1 leader_uuid: "9e87f62df5944bc199f0dac82a71e9fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } } }
I20260811 00:25:53.723006 4411 tablet_replica.cc:442] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:25:53.728399 4448 mvcc.cc:205] Tried to move back new op lower bound from 7317126978346532864 to 7317126978155909120. Current Snapshot: MvccSnapshot[applied={T|T < 7317126978346532864}]
I20260811 00:25:53.740060 4411 tablet_replica.cc:439] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e87f62df5944bc199f0dac82a71e9fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } } }
I20260811 00:25:53.740455 4411 tablet_replica.cc:442] T 17468bce420c4b619565a36c6cdbfbfb P 95af07ebb47248a6af9fa492eef17972: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:25:53.746003 4399 tablet_replica.cc:439] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e87f62df5944bc199f0dac82a71e9fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } } }
I20260811 00:25:53.746335 4399 tablet_replica.cc:442] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:25:53.756982 4395 tablet_replica.cc:439] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0: TxnStatusTablet state changed. Reason: New leader 9e87f62df5944bc199f0dac82a71e9fb. Latest consensus state: current_term: 1 leader_uuid: "9e87f62df5944bc199f0dac82a71e9fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } } }
I20260811 00:25:53.757387 4395 tablet_replica.cc:442] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:25:53.764448 4442 tablet_replica.cc:439] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e87f62df5944bc199f0dac82a71e9fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } } }
I20260811 00:25:53.764784 4442 tablet_replica.cc:442] T 17468bce420c4b619565a36c6cdbfbfb P 9e87f62df5944bc199f0dac82a71e9fb: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:25:53.765228 4411 consensus_queue.cc:1035] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:25:53.772974 4395 tablet_replica.cc:439] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e87f62df5944bc199f0dac82a71e9fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 } } peers { permanent_uuid: "95af07ebb47248a6af9fa492eef17972" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 36815 } } peers { permanent_uuid: "9e87f62df5944bc199f0dac82a71e9fb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41655 } } }
I20260811 00:25:53.773379 4395 tablet_replica.cc:442] T 17468bce420c4b619565a36c6cdbfbfb P 7ba1bd2b484e4d648a76ef97a1fcdbb0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:25:53.781608 4411 consensus_queue.cc:1035] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ba1bd2b484e4d648a76ef97a1fcdbb0" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 39645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:25:53.814934 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.794624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:47097 (local address 127.2.120.3:41655)
0811 00:25:53.794800 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:53.794806 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:53.805903 (+ 11097us) server_negotiation.cc:407] Connection header received
0811 00:25:53.806741 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:53.806748 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:53.806846 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:53.806917 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:53.807920 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.809808 (+ 1888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.812439 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.813168 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.813320 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.814001 (+ 681us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:53.814174 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:53.814277 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:53.814457 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 00:25:53.814559 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 00:25:53.815168 4463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.794025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:41655 (local address 127.2.120.2:47097)
0811 00:25:53.805752 (+ 11727us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:53.805784 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:53.806418 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:53.807188 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:53.807194 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:53.807213 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:25:53.807714 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:53.807731 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.810320 (+ 2589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.810369 (+ 49us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:53.812202 (+ 1833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:53.812229 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.813438 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.813448 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:53.813674 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.813764 (+ 90us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:25:53.814299 (+ 535us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:25:53.814313 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 00:25:53.814404 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 00:25:53.814584 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11480,"thread_start_us":11965,"threads_started":1}
I20260811 00:25:53.872375 4385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.863700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:46761 (local address 127.2.120.2:36815)
0811 00:25:53.864034 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:53.864040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:53.865877 (+ 1837us) server_negotiation.cc:407] Connection header received
0811 00:25:53.866115 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:53.866121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:53.866209 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:53.866313 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:53.866927 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.867926 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.870274 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.870708 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.870814 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.871566 (+ 752us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:53.871697 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:53.871819 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:53.871978 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 00:25:53.872071 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260811 00:25:53.872620 4466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.863464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:36815 (local address 127.2.120.3:46761)
0811 00:25:53.865724 (+ 2260us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:53.865747 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:53.865950 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:53.866330 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:53.866334 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:53.866347 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:25:53.866737 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:53.866748 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.868121 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.868127 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:53.870117 (+ 1990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:53.870135 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.870926 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.870933 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:53.871163 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.871246 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:25:53.871823 (+ 577us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:25:53.871838 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 00:25:53.871947 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 00:25:53.872148 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1929,"thread_start_us":1418,"threads_started":1}
I20260811 00:25:53.912792 3814 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:25:53.913077 3814 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:25:53.916651 3814 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b90fe2e2cb494eceb5c50925693b0858: Sending DeleteTablet for 3 replicas of tablet b14e3be81c8c419e99104b0aa094345a
I20260811 00:25:53.917665 3968 tablet_service.cc:1514] Processing DeleteTablet for tablet b14e3be81c8c419e99104b0aa094345a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:25:53 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:53356
I20260811 00:25:53.917701 4127 tablet_service.cc:1514] Processing DeleteTablet for tablet b14e3be81c8c419e99104b0aa094345a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:25:53 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:41132
I20260811 00:25:53.918231 4284 tablet_service.cc:1514] Processing DeleteTablet for tablet b14e3be81c8c419e99104b0aa094345a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:25:53 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:47550
I20260811 00:25:53.918594 4471 tablet_replica.cc:330] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972: stopping tablet replica
I20260811 00:25:53.918934 4471 raft_consensus.cc:2229] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:25:53.919564 4471 raft_consensus.cc:2258] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:25:53.921176 4471 ts_tablet_manager.cc:1905] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:25:53.929736 4472 tablet_replica.cc:330] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb: stopping tablet replica
I20260811 00:25:53.930305 4472 raft_consensus.cc:2229] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:25:53.930897 4472 raft_consensus.cc:2258] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:25:53.932855 4472 ts_tablet_manager.cc:1905] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:25:53.936128 4469 tablet_replica.cc:330] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0: stopping tablet replica
I20260811 00:25:53.936394 4469 raft_consensus.cc:2229] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:25:53.937023 4469 raft_consensus.cc:2258] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:25:53.938524 4469 ts_tablet_manager.cc:1905] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:25:53.941543 4471 ts_tablet_manager.cc:1918] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:25:53.944980 4471 log.cc:1196] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/b14e3be81c8c419e99104b0aa094345a
I20260811 00:25:53.945617 4471 ts_tablet_manager.cc:1939] T b14e3be81c8c419e99104b0aa094345a P 95af07ebb47248a6af9fa492eef17972: Deleting consensus metadata
I20260811 00:25:53.947108 3801 catalog_manager.cc:4841] TS 95af07ebb47248a6af9fa492eef17972 (127.2.120.2:36815): tablet b14e3be81c8c419e99104b0aa094345a (table test-table [id=a55e4c84969944fd9c6b7ad5a6360dff]) successfully deleted
I20260811 00:25:53.948390 4469 ts_tablet_manager.cc:1918] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:25:53.949296 4469 log.cc:1196] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/b14e3be81c8c419e99104b0aa094345a
I20260811 00:25:53.949987 4469 ts_tablet_manager.cc:1939] T b14e3be81c8c419e99104b0aa094345a P 7ba1bd2b484e4d648a76ef97a1fcdbb0: Deleting consensus metadata
Aug 11 00:25:53 dist-test-slave-fd7l krb5kdc[3764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407952, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:25:53.951378 3799 catalog_manager.cc:4841] TS 7ba1bd2b484e4d648a76ef97a1fcdbb0 (127.2.120.1:39645): tablet b14e3be81c8c419e99104b0aa094345a (table test-table [id=a55e4c84969944fd9c6b7ad5a6360dff]) successfully deleted
I20260811 00:25:53.951834 4472 ts_tablet_manager.cc:1918] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:25:53.952020 4472 log.cc:1196] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/b14e3be81c8c419e99104b0aa094345a
I20260811 00:25:53.952497 4472 ts_tablet_manager.cc:1939] T b14e3be81c8c419e99104b0aa094345a P 9e87f62df5944bc199f0dac82a71e9fb: Deleting consensus metadata
I20260811 00:25:53.953799 3795 catalog_manager.cc:4841] TS 9e87f62df5944bc199f0dac82a71e9fb (127.2.120.3:41655): tablet b14e3be81c8c419e99104b0aa094345a (table test-table [id=a55e4c84969944fd9c6b7ad5a6360dff]) successfully deleted
I20260811 00:25:53.960001 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.930604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45276 (local address 127.2.120.1:39645)
0811 00:25:53.930804 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:53.930813 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:53.937941 (+ 7128us) server_negotiation.cc:407] Connection header received
0811 00:25:53.938455 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:53.938463 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:53.938561 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:53.938667 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:53.940991 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.942290 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.944383 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.945289 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.945410 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.951780 (+ 6370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:53.951802 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:53.951827 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:53.951878 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:53.956200 (+ 4322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:53.957228 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:53.957241 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:53.957245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:53.957450 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:53.957856 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:53.957862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:53.957866 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:53.958355 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:53.958792 (+ 437us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:53.959190 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 00:25:53.959582 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 00:25:53.960948 4042 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45276
I20260811 00:25:53.984086 4480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.964498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47330 (local address 127.2.120.62:43815)
0811 00:25:53.967725 (+ 3227us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:53.967730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:53.967748 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:25:53.967808 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:53.967812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:53.967885 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:53.967985 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:53.969974 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.971013 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.973028 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.973559 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.973694 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.977324 (+ 3630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:53.977336 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:53.977340 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:53.977407 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:53.980497 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:53.981553 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:53.981559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:53.981562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:53.981742 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:53.982254 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:53.982259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:53.982263 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:53.982638 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:53.982877 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:53.983415 (+ 538us) server_negotiation.cc:299] Negotiation successful
0811 00:25:53.983690 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3118,"thread_start_us":2888,"threads_started":1}
W20260811 00:25:53.984799 3814 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47330
I20260811 00:25:53.987152 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 3896
I20260811 00:25:54.007658 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:53.988458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45290 (local address 127.2.120.1:39645)
0811 00:25:53.988687 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:53.988696 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:53.989160 (+ 464us) server_negotiation.cc:407] Connection header received
0811 00:25:53.989454 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:53.989461 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:53.989583 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:53.989703 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:53.991636 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.992814 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.994802 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:53.995630 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:53.995820 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:53.999382 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:53.999389 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:53.999395 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:53.999452 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:54.003757 (+ 4305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:54.005005 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:54.005012 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:54.005017 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:54.005197 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:54.005712 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:54.005717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:54.005721 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:54.006092 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:54.006380 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:54.006772 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 00:25:54.007123 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260811 00:25:54.008952 4042 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45290
W20260811 00:25:54.009760 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 3896: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:25:54.010099 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 3896
I20260811 00:25:54.041404 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 4052
Aug 11 00:25:54 dist-test-slave-fd7l krb5kdc[3764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407952, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.120.2@KRBTEST.COM
I20260811 00:25:54.059896 4385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:54.042261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40174 (local address 127.2.120.2:36815)
0811 00:25:54.042501 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:54.042508 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:54.042530 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:25:54.042639 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:54.042645 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:54.042724 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:54.042840 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:54.044171 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:54.045341 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:54.046834 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:54.047476 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:54.047651 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:54.052939 (+ 5288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:54.052959 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:54.052978 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:54.053028 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:54.056569 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:54.057477 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:54.057494 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:54.057496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:54.057709 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:54.058125 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:54.058131 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:54.058134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:54.058613 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:54.058970 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:54.059300 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 00:25:54.059558 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260811 00:25:54.061061 4201 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40174
W20260811 00:25:54.061864 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 4052: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:25:54.062075 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 4052
I20260811 00:25:54.101190 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 4211
Aug 11 00:25:54 dist-test-slave-fd7l krb5kdc[3764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407952, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.120.3@KRBTEST.COM
I20260811 00:25:54.118904 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:54.102181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58810 (local address 127.2.120.3:41655)
0811 00:25:54.102377 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:54.102383 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:54.102407 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:25:54.102536 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:54.102541 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:54.102640 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:54.102758 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:54.104065 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:54.104973 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:54.106229 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:54.106710 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:54.106864 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:54.111863 (+ 4999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:54.111879 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:54.111895 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:54.111936 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:54.115551 (+ 3615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:54.116377 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:54.116386 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:54.116389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:54.116609 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:54.117048 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:54.117054 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:54.117058 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:54.117532 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:54.117931 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:54.118228 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 00:25:54.118479 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 00:25:54.119748 4361 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58810
W20260811 00:25:54.120571 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 4211: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:25:54.120739 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 4211
I20260811 00:25:54.147876 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 3777
I20260811 00:25:54.149168 3861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:25:54.266260 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 3777
2026-08-11T00:25:54Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5535 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:25:54 dist-test-slave-fd7l krb5kdc[4485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:25:54 dist-test-slave-fd7l krb5kdc[4485](info): set up 2 sockets
Aug 11 00:25:54 dist-test-slave-fd7l krb5kdc[4485](info): commencing operation
krb5kdc: starting...
W20260811 00:25:55.474869 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.003s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:25:55 dist-test-slave-fd7l krb5kdc[4485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:25:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:25:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:25:55.712497 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:34195
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:34669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:34195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:56.065557 4500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:56.065860 4500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:56.065950 4500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:56.073170 4500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:25:56.073285 4500 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 00:25:56.073326 4500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:56.073381 4500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:56.073498 4500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:25:56.073598 4500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:25:56.083650 4500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:34669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:34195
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:34195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:56.085080 4500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:56.086377 4500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:56.098654 4509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:56.098703 4507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:56.098881 4506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:56.100001 4500 server_base.cc:1034] running on GCE node
I20260811 00:25:56.100890 4500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:56.102376 4500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:56.103636 4500 hybrid_clock.cc:648] HybridClock initialized: now 1786407956103578 us; error 38 us; skew 500 ppm
I20260811 00:25:56.103986 4500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:56 dist-test-slave-fd7l krb5kdc[4485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407956, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:56.111828 4500 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:25:56.113454 4500 webserver.cc:460] Webserver started at http://127.2.120.62:45753/ using document root <none> and password file <none>
I20260811 00:25:56.114157 4500 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:56.114274 4500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:56.114629 4500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:56.117718 4500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "5385c4e94d5e46648f9f6d41cd38ed8a"
format_stamp: "Formatted at 2026-08-11 00:25:56 on dist-test-slave-fd7l"
server_key: "d1da87b4d0fc80b68dece80e657c2c2a"
server_key_iv: "6ca40475f5a9068a6aae79eff33f68fb"
server_key_version: "encryptionkey@0"
I20260811 00:25:56.118497 4500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "5385c4e94d5e46648f9f6d41cd38ed8a"
format_stamp: "Formatted at 2026-08-11 00:25:56 on dist-test-slave-fd7l"
server_key: "d1da87b4d0fc80b68dece80e657c2c2a"
server_key_iv: "6ca40475f5a9068a6aae79eff33f68fb"
server_key_version: "encryptionkey@0"
I20260811 00:25:56.123960 4500 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 00:25:56.127431 4517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:56.129199 4500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:25:56.129401 4500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "5385c4e94d5e46648f9f6d41cd38ed8a"
format_stamp: "Formatted at 2026-08-11 00:25:56 on dist-test-slave-fd7l"
server_key: "d1da87b4d0fc80b68dece80e657c2c2a"
server_key_iv: "6ca40475f5a9068a6aae79eff33f68fb"
server_key_version: "encryptionkey@0"
I20260811 00:25:56.129676 4500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:56.141757 4500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:56.146507 4500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:56.146812 4500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:56.170584 4500 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:34195
I20260811 00:25:56.170619 4587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:34195 every 8 connection(s)
I20260811 00:25:56.172075 4500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:25:56.176056 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 4500
I20260811 00:25:56.176309 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:25:56.176927 4588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:56.177314 2528 external_mini_cluster.cc:1428] Setting key fbf0ad9efad6aa9ca7c6c2244f560600
I20260811 00:25:56.190923 4588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a: Bootstrap starting.
Aug 11 00:25:56 dist-test-slave-fd7l krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:25:56.194821 4588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:56.196195 4588 log.cc:824] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:56.199635 4588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a: No bootstrap required, opened a new log
I20260811 00:25:56.202574 4591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:56.179653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59844 (local address 127.2.120.62:34195)
0811 00:25:56.180744 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:56.180761 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:56.180810 (+ 49us) server_negotiation.cc:407] Connection header received
0811 00:25:56.182626 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:56.182652 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:56.183083 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:56.183554 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:56.185245 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:56.186524 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:56.187840 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:56.188716 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:56.188949 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:56.193569 (+ 4620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:56.193578 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:56.193595 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:56.193640 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:56.197981 (+ 4341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:56.198855 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:56.198862 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:56.198868 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:56.199068 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:56.199488 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:56.199504 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:56.199507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:56.200117 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:56.200471 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:56.200954 (+ 483us) server_negotiation.cc:299] Negotiation successful
0811 00:25:56.201439 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":436,"threads_started":1}
I20260811 00:25:56.208150 4588 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5385c4e94d5e46648f9f6d41cd38ed8a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 34195 } }
I20260811 00:25:56.208519 4588 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:56.208673 4588 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5385c4e94d5e46648f9f6d41cd38ed8a, State: Initialized, Role: FOLLOWER
I20260811 00:25:56.209313 4588 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5385c4e94d5e46648f9f6d41cd38ed8a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 34195 } }
I20260811 00:25:56.209646 4588 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:25:56.209807 4588 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:25:56.210011 4588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:56.212309 4588 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5385c4e94d5e46648f9f6d41cd38ed8a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 34195 } }
I20260811 00:25:56.212893 4588 leader_election.cc:302] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5385c4e94d5e46648f9f6d41cd38ed8a; no voters:
I20260811 00:25:56.213472 4588 leader_election.cc:288] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:25:56.213651 4593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:56.214422 4593 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a [term 1 LEADER]: Becoming Leader. State: Replica: 5385c4e94d5e46648f9f6d41cd38ed8a, State: Running, Role: LEADER
I20260811 00:25:56.214952 4593 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5385c4e94d5e46648f9f6d41cd38ed8a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 34195 } }
I20260811 00:25:56.215763 4588 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:25:56.218015 4594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5385c4e94d5e46648f9f6d41cd38ed8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5385c4e94d5e46648f9f6d41cd38ed8a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 34195 } } }
I20260811 00:25:56.218480 4594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:56.220218 4603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:25:56.220805 4595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5385c4e94d5e46648f9f6d41cd38ed8a. Latest consensus state: current_term: 1 leader_uuid: "5385c4e94d5e46648f9f6d41cd38ed8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5385c4e94d5e46648f9f6d41cd38ed8a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 34195 } } }
I20260811 00:25:56.221081 4595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a [sys.catalog]: This master's current role is: LEADER
I20260811 00:25:56.225253 4603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:25:56.234151 4603 catalog_manager.cc:1347] Generated new cluster ID: 517cc5c908d744b48425bbb408726e3f
I20260811 00:25:56.234279 4603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:25:56.255386 4603 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:25:56.256860 4603 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:25:56.266367 4603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a: Generated new TSK 0
I20260811 00:25:56.267284 4603 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
I20260811 00:25:56.322827 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:34195
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:34669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:56.684193 4617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:56.684568 4617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:56.684748 4617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:56.692209 4617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:56.692361 4617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:56.692510 4617 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 00:25:56.692631 4617 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:25:56.692804 4617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:25:56.704411 4617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:34669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.120.62:34195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:56.705983 4617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:56.707401 4617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:56.722818 4625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:56.722963 4627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:56.722771 4623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:56.724062 4617 server_base.cc:1034] running on GCE node
I20260811 00:25:56.725172 4617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:56.726438 4617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:56.727730 4617 hybrid_clock.cc:648] HybridClock initialized: now 1786407956727674 us; error 51 us; skew 500 ppm
I20260811 00:25:56.728142 4617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:56 dist-test-slave-fd7l krb5kdc[4485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407956, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:56.731844 4617 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:25:56.733619 4617 webserver.cc:460] Webserver started at http://127.2.120.1:45287/ using document root <none> and password file <none>
I20260811 00:25:56.734390 4617 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:56.734545 4617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:56.734853 4617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:56.738024 4617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "9e60fd992a124def95964b72fe153992"
format_stamp: "Formatted at 2026-08-11 00:25:56 on dist-test-slave-fd7l"
server_key: "58c5f01b4f2f7c8e76b2121c7b5310c7"
server_key_iv: "71b6063164f180f8afd555e963dd958a"
server_key_version: "encryptionkey@0"
I20260811 00:25:56.738770 4617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "9e60fd992a124def95964b72fe153992"
format_stamp: "Formatted at 2026-08-11 00:25:56 on dist-test-slave-fd7l"
server_key: "58c5f01b4f2f7c8e76b2121c7b5310c7"
server_key_iv: "71b6063164f180f8afd555e963dd958a"
server_key_version: "encryptionkey@0"
I20260811 00:25:56.745170 4617 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 00:25:56.749661 4635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:56.751377 4617 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:25:56.751564 4617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "9e60fd992a124def95964b72fe153992"
format_stamp: "Formatted at 2026-08-11 00:25:56 on dist-test-slave-fd7l"
server_key: "58c5f01b4f2f7c8e76b2121c7b5310c7"
server_key_iv: "71b6063164f180f8afd555e963dd958a"
server_key_version: "encryptionkey@0"
I20260811 00:25:56.751775 4617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:56.777216 4617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:56.782219 4617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:56.782646 4617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:56.785888 4617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:25:56.786038 4617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:56.786199 4617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:25:56.786314 4617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:25:56 dist-test-slave-fd7l krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407956, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:25:56.812415 4644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:56.786387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:44911 (local address 127.2.120.62:34195)
0811 00:25:56.786868 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:56.786877 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:56.788374 (+ 1497us) server_negotiation.cc:407] Connection header received
0811 00:25:56.790587 (+ 2213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:56.790594 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:56.790705 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:56.790835 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:56.793313 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:56.794723 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:56.797567 (+ 2844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:56.798015 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:56.798174 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:56.804093 (+ 5919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:56.804103 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:56.804109 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:56.804167 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:56.807481 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:56.808725 (+ 1244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:56.808732 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:56.808735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:56.808920 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:56.809733 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:56.809741 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:56.809744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:56.810071 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:56.810288 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:56.811394 (+ 1106us) server_negotiation.cc:299] Negotiation successful
0811 00:25:56.811767 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":252,"threads_started":1}
I20260811 00:25:56.812927 4646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:56.787256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:34195 (local address 127.2.120.1:44911)
0811 00:25:56.788188 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:56.788237 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:56.790250 (+ 2013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:56.791104 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:56.791114 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:56.791948 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:56.793020 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:56.793057 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:56.794942 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:56.794950 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:56.797371 (+ 2421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:56.797388 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:56.798314 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:56.798322 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:56.798552 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:56.799496 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:56.799551 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:56.803641 (+ 4090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:56.807730 (+ 4089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:56.807747 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:56.807777 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:56.808561 (+ 784us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:56.809103 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:56.809111 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:56.809118 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:56.809551 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:56.810386 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:56.810399 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:56.810868 (+ 469us) client_negotiation.cc:769] Sending connection context
0811 00:25:56.811147 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 00:25:56.811609 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":421,"threads_started":1}
I20260811 00:25:56.840883 4644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:56.823619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:52765 (local address 127.2.120.62:34195)
0811 00:25:56.823910 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:56.823920 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:56.825213 (+ 1293us) server_negotiation.cc:407] Connection header received
0811 00:25:56.825387 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:56.825396 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:56.825498 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:56.825633 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:56.827513 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:56.828256 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:56.830389 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:56.830858 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:56.831045 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:56.834136 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:56.834142 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:56.834146 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:56.834185 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:56.837476 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:56.838587 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:56.838597 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:56.838600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:56.838736 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:56.839157 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:56.839161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:56.839163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:56.839480 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:56.839666 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:56.840347 (+ 681us) server_negotiation.cc:299] Negotiation successful
0811 00:25:56.840637 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 00:25:56.841389 4729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:56.823469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:34195 (local address 127.2.120.1:52765)
0811 00:25:56.825060 (+ 1591us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:56.825084 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:56.825244 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:56.825715 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:56.825721 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:56.826700 (+ 979us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:56.827239 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:56.827261 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:56.828464 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:56.828469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:56.830187 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:56.830209 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:56.831085 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:56.831093 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:56.831288 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:56.832621 (+ 1333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:56.832672 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:56.833852 (+ 1180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:56.837684 (+ 3832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:56.837691 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:56.837696 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:56.838428 (+ 732us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:56.838866 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:56.838872 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:56.838875 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:56.839023 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:56.839636 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:56.839641 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:56.839938 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 00:25:56.840178 (+ 240us) client_negotiation.cc:240] Negotiation successful
0811 00:25:56.840718 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1427,"thread_start_us":1263,"threads_started":1}
I20260811 00:25:56.851545 4617 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:40929
I20260811 00:25:56.851580 4783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:40929 every 8 connection(s)
I20260811 00:25:56.853094 4617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:25:56.855336 4784 heartbeater.cc:347] Connected to a master server at 127.2.120.62:34195
I20260811 00:25:56.855558 4784 heartbeater.cc:464] Registering TS with master...
I20260811 00:25:56.856264 4784 heartbeater.cc:511] Master 127.2.120.62:34195 requested a full tablet report, sending...
I20260811 00:25:56.857352 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 4617
I20260811 00:25:56.857606 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:25:56.858315 2528 external_mini_cluster.cc:1428] Setting key 72efda31650556a45c98383651793aed
I20260811 00:25:56.858549 4539 ts_manager.cc:194] Registered new tserver with Master: 9e60fd992a124def95964b72fe153992 (127.2.120.1:40929)
I20260811 00:25:56.860846 4539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:44911
WARNING: no policy specified for kudu/127.2.120.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
I20260811 00:25:56.909116 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:34195
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:34669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:25:57.290295 4788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:57.290695 4788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:57.290879 4788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:57.299363 4788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:57.299540 4788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:57.299639 4788 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 00:25:57.299703 4788 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:25:57.299772 4788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:25:57.310655 4788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:34669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.120.62:34195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:57.312319 4788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:57.313956 4788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:57.331700 4793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:57.331619 4794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:57.331668 4796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:57.332644 4788 server_base.cc:1034] running on GCE node
I20260811 00:25:57.333508 4788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:57.334836 4788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:57.336124 4788 hybrid_clock.cc:648] HybridClock initialized: now 1786407957336026 us; error 92 us; skew 500 ppm
I20260811 00:25:57.336632 4788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:57 dist-test-slave-fd7l krb5kdc[4485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407957, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:57.340466 4788 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:25:57.342729 4788 webserver.cc:460] Webserver started at http://127.2.120.2:38999/ using document root <none> and password file <none>
I20260811 00:25:57.343559 4788 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:57.343719 4788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:57.344105 4788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:57.347842 4788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "0ef8379679be4d2fac916a81e4ea7162"
format_stamp: "Formatted at 2026-08-11 00:25:57 on dist-test-slave-fd7l"
server_key: "86f8c7eb69b22af18d31d11758e2ccd8"
server_key_iv: "3781d90523e50f403095b38081945e90"
server_key_version: "encryptionkey@0"
I20260811 00:25:57.348910 4788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "0ef8379679be4d2fac916a81e4ea7162"
format_stamp: "Formatted at 2026-08-11 00:25:57 on dist-test-slave-fd7l"
server_key: "86f8c7eb69b22af18d31d11758e2ccd8"
server_key_iv: "3781d90523e50f403095b38081945e90"
server_key_version: "encryptionkey@0"
I20260811 00:25:57.355270 4788 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:25:57.360052 4805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:57.361943 4788 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:25:57.362211 4788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "0ef8379679be4d2fac916a81e4ea7162"
format_stamp: "Formatted at 2026-08-11 00:25:57 on dist-test-slave-fd7l"
server_key: "86f8c7eb69b22af18d31d11758e2ccd8"
server_key_iv: "3781d90523e50f403095b38081945e90"
server_key_version: "encryptionkey@0"
I20260811 00:25:57.362470 4788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:57.394436 4788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:57.400398 4788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:57.400826 4788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:57.404367 4788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:25:57.404568 4788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:57.404708 4788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:25:57.404800 4788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:25:57 dist-test-slave-fd7l krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407957, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:25:57.437393 4817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:57.405335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:47961 (local address 127.2.120.62:34195)
0811 00:25:57.405995 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:57.406000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:57.407578 (+ 1578us) server_negotiation.cc:407] Connection header received
0811 00:25:57.410938 (+ 3360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:57.410948 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:57.411084 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:57.411188 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:57.414591 (+ 3403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:57.415566 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:57.419017 (+ 3451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:57.419796 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:57.419974 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:57.427156 (+ 7182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:57.427167 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:57.427175 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:57.427238 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:57.431870 (+ 4632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:57.433642 (+ 1772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:57.433650 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:57.433656 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:57.433844 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:57.434701 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:57.434706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:57.434708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:57.435030 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:57.435188 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:57.436725 (+ 1537us) server_negotiation.cc:299] Negotiation successful
0811 00:25:57.437070 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":420,"threads_started":1}
I20260811 00:25:57.439436 4818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:57.406284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:34195 (local address 127.2.120.2:47961)
0811 00:25:57.407353 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:57.407400 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:57.410167 (+ 2767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:57.411986 (+ 1819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:57.412003 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:57.413011 (+ 1008us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:57.414286 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:57.414318 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:57.415962 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:57.415967 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:57.418772 (+ 2805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:57.418794 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:57.420126 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:57.420138 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:57.420409 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:57.421465 (+ 1056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:57.421517 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:57.426679 (+ 5162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:57.432170 (+ 5491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:57.432191 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:57.432228 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:57.433352 (+ 1124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:57.434012 (+ 660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:57.434020 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:57.434027 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:57.434446 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:57.435203 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:57.435218 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:57.435949 (+ 731us) client_negotiation.cc:769] Sending connection context
0811 00:25:57.436416 (+ 467us) client_negotiation.cc:240] Negotiation successful
0811 00:25:57.437303 (+ 887us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"thread_start_us":482,"threads_started":1}
I20260811 00:25:57.464921 4817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:57.447564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:49457 (local address 127.2.120.62:34195)
0811 00:25:57.447852 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:57.447862 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:57.448380 (+ 518us) server_negotiation.cc:407] Connection header received
0811 00:25:57.448583 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:57.448592 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:57.448710 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:57.448833 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:57.451324 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:57.452399 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:57.453834 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:57.454352 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:57.454502 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:57.456705 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:57.456713 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:57.456718 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:57.456767 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:57.460322 (+ 3555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:57.461547 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:57.461556 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:57.461562 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:57.461728 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:57.462744 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:57.462755 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:57.462757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:57.463251 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:57.463482 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:57.464194 (+ 712us) server_negotiation.cc:299] Negotiation successful
0811 00:25:57.464491 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 00:25:57.465178 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:57.447498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:34195 (local address 127.2.120.2:49457)
0811 00:25:57.448251 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:57.448273 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:57.448424 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:57.448914 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:57.448921 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:57.450429 (+ 1508us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:57.451083 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:57.451105 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:57.452592 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:57.452605 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:57.453659 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:57.453673 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:57.454568 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:57.454574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:57.454752 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:57.455483 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:57.455502 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:57.456435 (+ 933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:57.460530 (+ 4095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:57.460540 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:57.460548 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:57.461308 (+ 760us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:57.462217 (+ 909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:57.462226 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:57.462235 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:57.462553 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:57.463422 (+ 869us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:57.463428 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:57.463678 (+ 250us) client_negotiation.cc:769] Sending connection context
0811 00:25:57.463997 (+ 319us) client_negotiation.cc:240] Negotiation successful
0811 00:25:57.464560 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":427,"threads_started":1}
I20260811 00:25:57.482940 4788 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:33923
I20260811 00:25:57.482996 4947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:33923 every 8 connection(s)
I20260811 00:25:57.484839 4788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:25:57.487825 4948 heartbeater.cc:347] Connected to a master server at 127.2.120.62:34195
I20260811 00:25:57.489388 4948 heartbeater.cc:464] Registering TS with master...
I20260811 00:25:57.489609 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 4788
I20260811 00:25:57.489882 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:25:57.490446 2528 external_mini_cluster.cc:1428] Setting key acd2edc1439800dba71bfb3d72c8e6f2
I20260811 00:25:57.490414 4948 heartbeater.cc:511] Master 127.2.120.62:34195 requested a full tablet report, sending...
I20260811 00:25:57.492353 4539 ts_manager.cc:194] Registered new tserver with Master: 0ef8379679be4d2fac916a81e4ea7162 (127.2.120.2:33923)
I20260811 00:25:57.493515 4539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:47961
WARNING: no policy specified for kudu/127.2.120.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
I20260811 00:25:57.548203 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:34195
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:34669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:25:57.863929 4784 heartbeater.cc:503] Master 127.2.120.62:34195 was elected leader, sending a full tablet report...
W20260811 00:25:57.956337 4952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:25:57.956777 4952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:25:57.956902 4952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:25:57.966388 4952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:25:57.966589 4952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:25:57.966703 4952 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 00:25:57.966764 4952 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:25:57.966948 4952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:25:57.978328 4952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:34669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.120.62:34195
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:25:57.979866 4952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:25:57.981379 4952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:25:57.996315 4958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:57.996287 4959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:25:57.996259 4961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:25:57.997313 4952 server_base.cc:1034] running on GCE node
I20260811 00:25:57.998044 4952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:25:57.999203 4952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:25:58.000762 4952 hybrid_clock.cc:648] HybridClock initialized: now 1786407958000570 us; error 61 us; skew 500 ppm
I20260811 00:25:58.001209 4952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:25:58 dist-test-slave-fd7l krb5kdc[4485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407958, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:25:58.005167 4952 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:25:58.007421 4952 webserver.cc:460] Webserver started at http://127.2.120.3:35965/ using document root <none> and password file <none>
I20260811 00:25:58.008620 4952 fs_manager.cc:359] Metadata directory not provided
I20260811 00:25:58.008805 4952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:25:58.009217 4952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:25:58.012925 4952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "ae35798ed19f48dca0783ea23994c706"
format_stamp: "Formatted at 2026-08-11 00:25:58 on dist-test-slave-fd7l"
server_key: "b0bc245a335a22dd047939b8b6616767"
server_key_iv: "a1cc9fe854a6303c11fac48a28516c88"
server_key_version: "encryptionkey@0"
I20260811 00:25:58.013871 4952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "ae35798ed19f48dca0783ea23994c706"
format_stamp: "Formatted at 2026-08-11 00:25:58 on dist-test-slave-fd7l"
server_key: "b0bc245a335a22dd047939b8b6616767"
server_key_iv: "a1cc9fe854a6303c11fac48a28516c88"
server_key_version: "encryptionkey@0"
I20260811 00:25:58.021023 4952 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260811 00:25:58.025482 4968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:58.027055 4952 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:25:58.027274 4952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "ae35798ed19f48dca0783ea23994c706"
format_stamp: "Formatted at 2026-08-11 00:25:58 on dist-test-slave-fd7l"
server_key: "b0bc245a335a22dd047939b8b6616767"
server_key_iv: "a1cc9fe854a6303c11fac48a28516c88"
server_key_version: "encryptionkey@0"
I20260811 00:25:58.027536 4952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:25:58.042738 4952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:25:58.048053 4952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:25:58.048440 4952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:25:58.051482 4952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:25:58.051797 4952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:25:58.052008 4952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:25:58.052158 4952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:25:58 dist-test-slave-fd7l krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407958, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:25:58.083015 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.052408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:57295 (local address 127.2.120.62:34195)
0811 00:25:58.053240 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:58.053248 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:58.054495 (+ 1247us) server_negotiation.cc:407] Connection header received
0811 00:25:58.057302 (+ 2807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:58.057313 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:58.057466 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:58.057626 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:58.060237 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.061393 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.064370 (+ 2977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.064897 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.065052 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.070541 (+ 5489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:58.070554 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:58.070563 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:58.070647 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:58.075618 (+ 4971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:58.077824 (+ 2206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:58.077836 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:58.077838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:58.078249 (+ 411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:58.079666 (+ 1417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:58.079674 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:58.079681 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:58.080327 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:58.080664 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:58.082370 (+ 1706us) server_negotiation.cc:299] Negotiation successful
0811 00:25:58.082653 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":559,"threads_started":1}
I20260811 00:25:58.085954 4983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.053171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:34195 (local address 127.2.120.3:57295)
0811 00:25:58.054256 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:58.054321 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:58.056896 (+ 2575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:58.057932 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:58.057947 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:58.058829 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:58.059979 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.060032 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.061594 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.061607 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.064151 (+ 2544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.064175 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.065204 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.065214 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.065460 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.066494 (+ 1034us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:58.066542 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:58.070106 (+ 3564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:58.076048 (+ 5942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:58.076064 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:58.076090 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:58.077580 (+ 1490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:58.078575 (+ 995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:58.078583 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:58.078587 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:58.079408 (+ 821us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:58.080684 (+ 1276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:58.080701 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:58.081562 (+ 861us) client_negotiation.cc:769] Sending connection context
0811 00:25:58.082172 (+ 610us) client_negotiation.cc:240] Negotiation successful
0811 00:25:58.083208 (+ 1036us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"thread_start_us":557,"threads_started":1}
I20260811 00:25:58.112773 4952 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:38583
I20260811 00:25:58.112785 5105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:38583 every 8 connection(s)
I20260811 00:25:58.114302 4952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:25:58.116412 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.095693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:43091 (local address 127.2.120.62:34195)
0811 00:25:58.096010 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:58.096023 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:58.097128 (+ 1105us) server_negotiation.cc:407] Connection header received
0811 00:25:58.097490 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:58.097496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:58.097604 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:58.097765 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:58.099751 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.101035 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.103811 (+ 2776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.104743 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.104959 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.108316 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:58.108343 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:58.108348 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:58.108411 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:58.112488 (+ 4077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:58.113637 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:58.113651 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:58.113654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:58.113821 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:58.114289 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:58.114294 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:58.114298 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:58.114722 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:58.114914 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:58.115587 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 00:25:58.115874 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260811 00:25:58.117014 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 4952
I20260811 00:25:58.117241 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:25:58.117933 2528 external_mini_cluster.cc:1428] Setting key 9a960e70197008f72e5313929c4b4d4d
I20260811 00:25:58.118242 5107 heartbeater.cc:347] Connected to a master server at 127.2.120.62:34195
I20260811 00:25:58.120661 5107 heartbeater.cc:464] Registering TS with master...
I20260811 00:25:58.122326 5107 heartbeater.cc:511] Master 127.2.120.62:34195 requested a full tablet report, sending...
I20260811 00:25:58.132318 5066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.095518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:34195 (local address 127.2.120.3:43091)
0811 00:25:58.096950 (+ 1432us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:58.096979 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:58.097228 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:58.097844 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:58.097849 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:58.098791 (+ 942us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:58.099466 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.099499 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.101235 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.101242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.103459 (+ 2217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.103496 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.105091 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.105102 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.105386 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.106417 (+ 1031us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:58.106437 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:58.107921 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:58.112788 (+ 4867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:58.112796 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:58.112801 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:58.113444 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:58.113980 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:58.113985 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:58.113990 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:58.114152 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:58.114877 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:58.114883 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:58.115248 (+ 365us) client_negotiation.cc:769] Sending connection context
0811 00:25:58.115455 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 00:25:58.115894 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1204,"thread_start_us":929,"threads_started":1}
I20260811 00:25:58.132592 4539 ts_manager.cc:194] Registered new tserver with Master: ae35798ed19f48dca0783ea23994c706 (127.2.120.3:38583)
I20260811 00:25:58.133917 4539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:57295
I20260811 00:25:58.139349 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 00:25:58 dist-test-slave-fd7l krb5kdc[4485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 00:25:58 dist-test-slave-fd7l krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:25:58.206404 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.186895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59860 (local address 127.2.120.62:34195)
0811 00:25:58.187145 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:58.187154 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:58.189269 (+ 2115us) server_negotiation.cc:407] Connection header received
0811 00:25:58.189495 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:58.189500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:58.189600 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:58.189717 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:58.191476 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.192212 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.193548 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.193954 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.194105 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.198701 (+ 4596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:58.198709 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:58.198713 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:58.198776 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:58.202835 (+ 4059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:58.204184 (+ 1349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:58.204189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:58.204191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:58.204374 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:58.204948 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:58.204951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:58.204954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:58.205264 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:58.205481 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:58.205910 (+ 429us) server_negotiation.cc:299] Negotiation successful
0811 00:25:58.206120 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 00:25:58.211547 4539 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:25:58.220638 4539 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:25:58.250289 5124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.235860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:38583 (local address 127.0.0.1:40020)
0811 00:25:58.239995 (+ 4135us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:58.240019 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:58.240153 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:58.241489 (+ 1336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:58.241496 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:58.241527 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:58.242066 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.242080 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.244477 (+ 2397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.244483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.246649 (+ 2166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.246668 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.249144 (+ 2476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.249151 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.249427 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.249496 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 00:25:58.249638 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 00:25:58.249869 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3963,"thread_start_us":3812,"threads_started":1}
I20260811 00:25:58.251478 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.236784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40020 (local address 127.2.120.3:38583)
0811 00:25:58.238535 (+ 1751us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:58.238546 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:58.240292 (+ 1746us) server_negotiation.cc:407] Connection header received
0811 00:25:58.240439 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:58.240448 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:58.240771 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:58.241110 (+ 339us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:58.242430 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.244284 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.246855 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.248583 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.248716 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.248975 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:58.250915 (+ 1940us) server_negotiation.cc:299] Negotiation successful
0811 00:25:58.251127 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1557,"thread_start_us":1874,"threads_started":1}
I20260811 00:25:58.251811 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.235751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:40929 (local address 127.0.0.1:38716)
0811 00:25:58.236589 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:58.236637 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:58.236849 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:58.238954 (+ 2105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:58.238962 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:58.239004 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:58.239454 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.239468 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.243488 (+ 4020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.243494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.247320 (+ 3826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.247335 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.249405 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.249412 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.251097 (+ 1685us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.251130 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 00:25:58.251200 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 00:25:58.251352 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":1601,"threads_started":1}
I20260811 00:25:58.252988 5125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.236399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38716 (local address 127.2.120.1:40929)
0811 00:25:58.237453 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:58.237462 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:58.237493 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:25:58.237603 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:58.237611 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:58.237859 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:58.239711 (+ 1852us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:58.239940 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.243298 (+ 3358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.247616 (+ 4318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.249156 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.252320 (+ 3164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.252502 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:58.252639 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 00:25:58.252774 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":902,"thread_start_us":778,"threads_started":1}
I20260811 00:25:58.257634 5029 tablet_service.cc:1467] Processing CreateTablet for tablet de9f5babac9440c487193cb0b9ae6405 (DEFAULT_TABLE table=test-table [id=35d15ab82a5b4e0ebab18fbfaeb074e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:25:58.257637 4697 tablet_service.cc:1467] Processing CreateTablet for tablet de9f5babac9440c487193cb0b9ae6405 (DEFAULT_TABLE table=test-table [id=35d15ab82a5b4e0ebab18fbfaeb074e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:25:58.259469 5029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de9f5babac9440c487193cb0b9ae6405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:58.259713 4697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de9f5babac9440c487193cb0b9ae6405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:58.261572 5124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.242286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:33923 (local address 127.0.0.1:41748)
0811 00:25:58.250586 (+ 8300us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:58.250611 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:58.250730 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:58.254080 (+ 3350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:58.254086 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:58.254118 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:58.254306 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.254318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.256393 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.256401 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.258464 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.258487 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.260670 (+ 2183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.260678 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.260898 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.260951 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 00:25:58.261050 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 00:25:58.261227 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8182,"thread_start_us":11160,"threads_started":1}
I20260811 00:25:58.262665 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.240556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41748 (local address 127.2.120.2:33923)
0811 00:25:58.253008 (+ 12452us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:58.253022 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:58.253068 (+ 46us) server_negotiation.cc:407] Connection header received
0811 00:25:58.253231 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:58.253245 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:58.253583 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:58.253844 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:58.254507 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.256181 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.258733 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.260298 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.260453 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.262036 (+ 1583us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:58.262126 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 00:25:58.262333 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12253,"thread_start_us":12111,"threads_started":1}
I20260811 00:25:58.267310 4866 tablet_service.cc:1467] Processing CreateTablet for tablet de9f5babac9440c487193cb0b9ae6405 (DEFAULT_TABLE table=test-table [id=35d15ab82a5b4e0ebab18fbfaeb074e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:25:58.269145 4866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de9f5babac9440c487193cb0b9ae6405. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:58.284411 5133 tablet_bootstrap.cc:492] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992: Bootstrap starting.
I20260811 00:25:58.285738 5132 tablet_bootstrap.cc:492] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706: Bootstrap starting.
I20260811 00:25:58.288564 5133 tablet_bootstrap.cc:654] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:58.290782 5133 log.cc:824] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:58.291035 5132 tablet_bootstrap.cc:654] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:58.292935 5132 log.cc:824] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:58.295624 5136 tablet_bootstrap.cc:492] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162: Bootstrap starting.
I20260811 00:25:58.304801 5136 tablet_bootstrap.cc:654] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:58.305716 5133 tablet_bootstrap.cc:492] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992: No bootstrap required, opened a new log
I20260811 00:25:58.306061 5133 ts_tablet_manager.cc:1397] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992: Time spent bootstrapping tablet: real 0.022s user 0.008s sys 0.011s
I20260811 00:25:58.306437 5136 log.cc:824] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162: Log is configured to *not* fsync() on all Append() calls
I20260811 00:25:58.310037 5132 tablet_bootstrap.cc:492] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706: No bootstrap required, opened a new log
I20260811 00:25:58.310519 5136 tablet_bootstrap.cc:492] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162: No bootstrap required, opened a new log
I20260811 00:25:58.310536 5132 ts_tablet_manager.cc:1397] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.008s
I20260811 00:25:58.310842 5136 ts_tablet_manager.cc:1397] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.003s
I20260811 00:25:58.317523 5133 raft_consensus.cc:348] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } }
I20260811 00:25:58.318082 5133 raft_consensus.cc:374] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:58.318186 5133 raft_consensus.cc:729] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e60fd992a124def95964b72fe153992, State: Initialized, Role: FOLLOWER
I20260811 00:25:58.319626 5133 consensus_queue.cc:260] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } }
I20260811 00:25:58.324815 5136 raft_consensus.cc:348] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } }
I20260811 00:25:58.325428 5136 raft_consensus.cc:374] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:58.325570 5136 raft_consensus.cc:729] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ef8379679be4d2fac916a81e4ea7162, State: Initialized, Role: FOLLOWER
I20260811 00:25:58.326565 5136 consensus_queue.cc:260] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } }
I20260811 00:25:58.327471 5132 raft_consensus.cc:348] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } }
I20260811 00:25:58.328053 5132 raft_consensus.cc:374] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:58.328163 5133 ts_tablet_manager.cc:1428] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992: Time spent starting tablet: real 0.022s user 0.010s sys 0.010s
I20260811 00:25:58.328220 5132 raft_consensus.cc:729] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae35798ed19f48dca0783ea23994c706, State: Initialized, Role: FOLLOWER
I20260811 00:25:58.329233 5132 consensus_queue.cc:260] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } }
I20260811 00:25:58.335158 5136 ts_tablet_manager.cc:1428] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162: Time spent starting tablet: real 0.024s user 0.018s sys 0.007s
I20260811 00:25:58.335836 4948 heartbeater.cc:503] Master 127.2.120.62:34195 was elected leader, sending a full tablet report...
I20260811 00:25:58.338760 5107 heartbeater.cc:503] Master 127.2.120.62:34195 was elected leader, sending a full tablet report...
I20260811 00:25:58.341042 5132 ts_tablet_manager.cc:1428] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706: Time spent starting tablet: real 0.030s user 0.012s sys 0.013s
W20260811 00:25:58.356045 4785 tablet.cc:2369] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:25:58.371188 5108 tablet.cc:2369] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:25:58.474993 5142 raft_consensus.cc:482] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:25:58.475370 5142 raft_consensus.cc:504] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } }
I20260811 00:25:58.477505 5142 leader_election.cc:288] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae35798ed19f48dca0783ea23994c706 (127.2.120.3:38583), 0ef8379679be4d2fac916a81e4ea7162 (127.2.120.2:33923)
W20260811 00:25:58.488092 4949 tablet.cc:2369] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:25:58.488297 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.478243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:41133 (local address 127.2.120.3:38583)
0811 00:25:58.478560 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:58.478571 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:58.479438 (+ 867us) server_negotiation.cc:407] Connection header received
0811 00:25:58.479568 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:58.479576 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:58.479689 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:58.479817 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:58.480940 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.482273 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.485690 (+ 3417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.487021 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.487140 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.487261 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:58.487881 (+ 620us) server_negotiation.cc:299] Negotiation successful
0811 00:25:58.488055 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 00:25:58.489212 5145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.477917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:38583 (local address 127.2.120.1:41133)
0811 00:25:58.479153 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:58.479177 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:58.479346 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:58.480237 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:58.480242 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:58.480278 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:58.480767 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.480780 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.482524 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.482530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.485434 (+ 2904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.485456 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.487476 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.487483 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.487670 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.487706 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 00:25:58.488614 (+ 908us) client_negotiation.cc:240] Negotiation successful
0811 00:25:58.488802 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1059,"mutex_wait_us":100,"thread_start_us":860,"threads_started":1}
I20260811 00:25:58.490500 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.479679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:52659 (local address 127.2.120.2:33923)
0811 00:25:58.480200 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:58.480209 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:58.480415 (+ 206us) server_negotiation.cc:407] Connection header received
0811 00:25:58.480662 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:58.480677 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:58.480781 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:58.480898 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:58.481626 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.483179 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.486926 (+ 3747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.488663 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.488804 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.488944 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:58.490060 (+ 1116us) server_negotiation.cc:299] Negotiation successful
0811 00:25:58.490223 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319}
I20260811 00:25:58.490584 5050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de9f5babac9440c487193cb0b9ae6405" candidate_uuid: "9e60fd992a124def95964b72fe153992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae35798ed19f48dca0783ea23994c706" is_pre_election: true
I20260811 00:25:58.491217 5050 raft_consensus.cc:2454] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e60fd992a124def95964b72fe153992 in term 0.
I20260811 00:25:58.491798 5146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.479360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:33923 (local address 127.2.120.1:52659)
0811 00:25:58.480288 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:58.480309 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:58.480472 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:58.480975 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:58.480980 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:58.481013 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:58.481410 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.481424 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.483599 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.483608 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.486648 (+ 3040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.486673 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.489616 (+ 2943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.489622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.489846 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.489881 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 00:25:58.490798 (+ 917us) client_negotiation.cc:240] Negotiation successful
0811 00:25:58.491157 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":795,"mutex_wait_us":35,"thread_start_us":596,"threads_started":1}
I20260811 00:25:58.492061 4639 leader_election.cc:302] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e60fd992a124def95964b72fe153992, ae35798ed19f48dca0783ea23994c706; no voters:
I20260811 00:25:58.492558 5142 raft_consensus.cc:2781] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:25:58.492749 5142 raft_consensus.cc:482] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:25:58.492877 5142 raft_consensus.cc:3037] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:58.493378 4890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de9f5babac9440c487193cb0b9ae6405" candidate_uuid: "9e60fd992a124def95964b72fe153992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef8379679be4d2fac916a81e4ea7162" is_pre_election: true
I20260811 00:25:58.493957 4890 raft_consensus.cc:2454] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e60fd992a124def95964b72fe153992 in term 0.
I20260811 00:25:58.497562 5142 raft_consensus.cc:504] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } }
I20260811 00:25:58.498674 5142 leader_election.cc:288] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [CANDIDATE]: Term 1 election: Requested vote from peers ae35798ed19f48dca0783ea23994c706 (127.2.120.3:38583), 0ef8379679be4d2fac916a81e4ea7162 (127.2.120.2:33923)
I20260811 00:25:58.499334 5050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de9f5babac9440c487193cb0b9ae6405" candidate_uuid: "9e60fd992a124def95964b72fe153992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae35798ed19f48dca0783ea23994c706"
I20260811 00:25:58.499625 5050 raft_consensus.cc:3037] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:58.499677 4890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de9f5babac9440c487193cb0b9ae6405" candidate_uuid: "9e60fd992a124def95964b72fe153992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef8379679be4d2fac916a81e4ea7162"
I20260811 00:25:58.499929 4890 raft_consensus.cc:3037] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:58.502592 4890 raft_consensus.cc:2454] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e60fd992a124def95964b72fe153992 in term 1.
I20260811 00:25:58.503202 4639 leader_election.cc:302] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ef8379679be4d2fac916a81e4ea7162, 9e60fd992a124def95964b72fe153992; no voters:
I20260811 00:25:58.503553 5142 raft_consensus.cc:2781] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:58.504145 5142 raft_consensus.cc:686] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [term 1 LEADER]: Becoming Leader. State: Replica: 9e60fd992a124def95964b72fe153992, State: Running, Role: LEADER
I20260811 00:25:58.504617 5050 raft_consensus.cc:2454] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e60fd992a124def95964b72fe153992 in term 1.
I20260811 00:25:58.504782 5142 consensus_queue.cc:237] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } }
I20260811 00:25:58.509847 4538 catalog_manager.cc:5495] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e60fd992a124def95964b72fe153992 (127.2.120.1). New cstate: current_term: 1 leader_uuid: "9e60fd992a124def95964b72fe153992" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } health_report { overall_health: UNKNOWN } } }
Aug 11 00:25:58 dist-test-slave-fd7l krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407956, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:25:58.583942 5157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.567431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:34195 (local address 127.0.0.1:59874)
0811 00:25:58.568206 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:58.568225 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:58.568427 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:58.568838 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:58.568845 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:58.569610 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:25:58.569889 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.569899 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.570988 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.570998 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.571984 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.571997 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.572874 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.572880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.573057 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.574080 (+ 1023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:25:58.574100 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:25:58.577356 (+ 3256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:25:58.580784 (+ 3428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:58.580802 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:58.580810 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:58.581380 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:58.581927 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:25:58.581931 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:25:58.581934 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:25:58.582107 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:25:58.582795 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:25:58.582808 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:25:58.583028 (+ 220us) client_negotiation.cc:769] Sending connection context
0811 00:25:58.583235 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 00:25:58.583522 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":457,"threads_started":1}
I20260811 00:25:58.584672 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.567431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59874 (local address 127.2.120.62:34195)
0811 00:25:58.567666 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:58.567671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:58.568353 (+ 682us) server_negotiation.cc:407] Connection header received
0811 00:25:58.568599 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:58.568605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:58.568676 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:58.568793 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:25:58.570061 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.570825 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.572165 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.572676 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.572819 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.577631 (+ 4812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:25:58.577637 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:25:58.577641 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:25:58.577682 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:25:58.580603 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:58.581529 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:58.581534 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:58.581536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:58.581769 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:25:58.582259 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:25:58.582263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:25:58.582265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:25:58.582578 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:25:58.582698 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:58.583554 (+ 856us) server_negotiation.cc:299] Negotiation successful
0811 00:25:58.584035 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260811 00:25:58.590832 4538 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:59874:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 00:25:58.592092 4538 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:25:58.599393 4697 tablet_service.cc:1467] Processing CreateTablet for tablet 5a7e81fa8b7643bf887c6007c14c045d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=473ba11d0fee4194b3b18e80f9805c41]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:25:58.599411 5029 tablet_service.cc:1467] Processing CreateTablet for tablet 5a7e81fa8b7643bf887c6007c14c045d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=473ba11d0fee4194b3b18e80f9805c41]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:25:58.599387 4866 tablet_service.cc:1467] Processing CreateTablet for tablet 5a7e81fa8b7643bf887c6007c14c045d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=473ba11d0fee4194b3b18e80f9805c41]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:25:58.600404 4697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a7e81fa8b7643bf887c6007c14c045d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:58.600407 5029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a7e81fa8b7643bf887c6007c14c045d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:58.600407 4866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a7e81fa8b7643bf887c6007c14c045d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:58.607019 5133 tablet_bootstrap.cc:492] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992: Bootstrap starting.
I20260811 00:25:58.608094 5132 tablet_bootstrap.cc:492] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706: Bootstrap starting.
I20260811 00:25:58.608182 5136 tablet_bootstrap.cc:492] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162: Bootstrap starting.
I20260811 00:25:58.608800 5133 tablet_bootstrap.cc:654] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:58.610025 5132 tablet_bootstrap.cc:654] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:58.610040 5136 tablet_bootstrap.cc:654] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:58.612282 5133 tablet_bootstrap.cc:492] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992: No bootstrap required, opened a new log
I20260811 00:25:58.612536 5133 ts_tablet_manager.cc:1397] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260811 00:25:58.613725 5133 raft_consensus.cc:348] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:25:58.614105 5133 raft_consensus.cc:374] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:58.614217 5133 raft_consensus.cc:729] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e60fd992a124def95964b72fe153992, State: Initialized, Role: FOLLOWER
I20260811 00:25:58.614560 5133 consensus_queue.cc:260] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:25:58.615432 5133 ts_tablet_manager.cc:1428] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 00:25:58.615182 5142 tablet_replica.cc:439] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } }
I20260811 00:25:58.616292 5142 tablet_replica.cc:442] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:25:58.617275 5132 tablet_bootstrap.cc:492] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706: No bootstrap required, opened a new log
I20260811 00:25:58.617461 5132 ts_tablet_manager.cc:1397] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260811 00:25:58.617767 5136 tablet_bootstrap.cc:492] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162: No bootstrap required, opened a new log
I20260811 00:25:58.617938 5136 ts_tablet_manager.cc:1397] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 00:25:58.618435 5132 raft_consensus.cc:348] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:25:58.618736 5132 raft_consensus.cc:374] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:58.618860 5132 raft_consensus.cc:729] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae35798ed19f48dca0783ea23994c706, State: Initialized, Role: FOLLOWER
I20260811 00:25:58.618789 5136 raft_consensus.cc:348] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:25:58.619072 5136 raft_consensus.cc:374] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:58.619184 5136 raft_consensus.cc:729] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ef8379679be4d2fac916a81e4ea7162, State: Initialized, Role: FOLLOWER
I20260811 00:25:58.619133 5132 consensus_queue.cc:260] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:25:58.619498 5136 consensus_queue.cc:260] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:25:58.619784 5132 ts_tablet_manager.cc:1428] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 00:25:58.620143 5136 ts_tablet_manager.cc:1428] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260811 00:25:58.620451 5143 tablet_replica.cc:439] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } }
I20260811 00:25:58.620944 5143 tablet_replica.cc:442] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:25:58.621737 5144 tablet_replica.cc:439] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } }
I20260811 00:25:58.622072 5144 tablet_replica.cc:442] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:25:58.816490 5144 raft_consensus.cc:482] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:25:58.816862 5144 raft_consensus.cc:504] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:25:58.818861 5144 leader_election.cc:288] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e60fd992a124def95964b72fe153992 (127.2.120.1:40929), 0ef8379679be4d2fac916a81e4ea7162 (127.2.120.2:33923)
I20260811 00:25:58.826704 5163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.819407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:55559 (local address 127.2.120.1:40929)
0811 00:25:58.820401 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:58.820411 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:58.820437 (+ 26us) server_negotiation.cc:407] Connection header received
0811 00:25:58.820577 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:58.820584 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:58.820692 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:58.820812 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:58.821437 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.822380 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.824427 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.825578 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.825699 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.825784 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:58.826185 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 00:25:58.826350 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":840,"thread_start_us":632,"threads_started":1}
I20260811 00:25:58.826715 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.819206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:40929 (local address 127.2.120.3:55559)
0811 00:25:58.819818 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:58.819843 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:58.820048 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:58.820886 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:58.820892 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:58.820936 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:58.821221 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.821236 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.822544 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.822550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.824205 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.824220 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.825778 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.825786 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.825976 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.826025 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 00:25:58.826125 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 00:25:58.826297 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":71,"thread_start_us":388,"threads_started":1}
I20260811 00:25:58.827117 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.819527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:43593 (local address 127.2.120.2:33923)
0811 00:25:58.819770 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:58.819778 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:58.820824 (+ 1046us) server_negotiation.cc:407] Connection header received
0811 00:25:58.820963 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:58.820971 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:58.821055 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:58.821189 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:25:58.821851 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.822742 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.825035 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.825950 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.826121 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.826216 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:58.826649 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 00:25:58.826812 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 00:25:58.827173 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.819206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:33923 (local address 127.2.120.3:43593)
0811 00:25:58.820658 (+ 1452us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:58.820687 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:58.820838 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:58.821226 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:58.821229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:58.821251 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:25:58.821660 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.821672 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.822928 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.822934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.824853 (+ 1919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.824874 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.826237 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.826245 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.826423 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.826469 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 00:25:58.826592 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 00:25:58.826761 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1272,"mutex_wait_us":221,"thread_start_us":842,"threads_started":1}
I20260811 00:25:58.827997 4890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a7e81fa8b7643bf887c6007c14c045d" candidate_uuid: "ae35798ed19f48dca0783ea23994c706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef8379679be4d2fac916a81e4ea7162" is_pre_election: true
I20260811 00:25:58.828284 4890 raft_consensus.cc:2454] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae35798ed19f48dca0783ea23994c706 in term 0.
I20260811 00:25:58.828263 4723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a7e81fa8b7643bf887c6007c14c045d" candidate_uuid: "ae35798ed19f48dca0783ea23994c706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e60fd992a124def95964b72fe153992" is_pre_election: true
I20260811 00:25:58.828755 4723 raft_consensus.cc:2454] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae35798ed19f48dca0783ea23994c706 in term 0.
I20260811 00:25:58.828881 4972 leader_election.cc:302] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ef8379679be4d2fac916a81e4ea7162, ae35798ed19f48dca0783ea23994c706; no voters:
I20260811 00:25:58.829418 5144 raft_consensus.cc:2781] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:25:58.829581 5144 raft_consensus.cc:482] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:25:58.829706 5144 raft_consensus.cc:3037] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:58.832013 5144 raft_consensus.cc:504] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:25:58.832801 5144 leader_election.cc:288] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [CANDIDATE]: Term 1 election: Requested vote from peers 9e60fd992a124def95964b72fe153992 (127.2.120.1:40929), 0ef8379679be4d2fac916a81e4ea7162 (127.2.120.2:33923)
I20260811 00:25:58.833560 4890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a7e81fa8b7643bf887c6007c14c045d" candidate_uuid: "ae35798ed19f48dca0783ea23994c706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef8379679be4d2fac916a81e4ea7162"
I20260811 00:25:58.833832 4890 raft_consensus.cc:3037] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:58.836028 4890 raft_consensus.cc:2454] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae35798ed19f48dca0783ea23994c706 in term 1.
I20260811 00:25:58.836557 4723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a7e81fa8b7643bf887c6007c14c045d" candidate_uuid: "ae35798ed19f48dca0783ea23994c706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e60fd992a124def95964b72fe153992"
I20260811 00:25:58.836834 4723 raft_consensus.cc:3037] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:58.836867 4972 leader_election.cc:302] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ef8379679be4d2fac916a81e4ea7162, ae35798ed19f48dca0783ea23994c706; no voters:
I20260811 00:25:58.837504 5144 raft_consensus.cc:2781] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:58.838060 5144 raft_consensus.cc:686] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [term 1 LEADER]: Becoming Leader. State: Replica: ae35798ed19f48dca0783ea23994c706, State: Running, Role: LEADER
I20260811 00:25:58.838716 5144 consensus_queue.cc:237] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:25:58.840621 4723 raft_consensus.cc:2454] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae35798ed19f48dca0783ea23994c706 in term 1.
I20260811 00:25:58.843005 5165 tablet_replica.cc:439] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706: TxnStatusTablet state changed. Reason: New leader ae35798ed19f48dca0783ea23994c706. Latest consensus state: current_term: 1 leader_uuid: "ae35798ed19f48dca0783ea23994c706" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } }
I20260811 00:25:58.843457 5165 tablet_replica.cc:442] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:25:58.844269 5167 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 00:25:58.844136 4536 catalog_manager.cc:5495] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae35798ed19f48dca0783ea23994c706 (127.2.120.3). New cstate: current_term: 1 leader_uuid: "ae35798ed19f48dca0783ea23994c706" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } health_report { overall_health: HEALTHY } } }
I20260811 00:25:58.844600 5167 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 00:25:58.917006 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.909883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40030 (local address 127.2.120.3:38583)
0811 00:25:58.910145 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:58.910151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:58.910297 (+ 146us) server_negotiation.cc:407] Connection header received
0811 00:25:58.910586 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:58.910590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:58.910691 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:58.910865 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:58.911287 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.912207 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.914200 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.914754 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.914860 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.915538 (+ 678us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:58.916204 (+ 666us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:58.916341 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:58.916610 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 00:25:58.916717 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 00:25:58.917114 5157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:58.909790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:38583 (local address 127.0.0.1:40030)
0811 00:25:58.910197 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:58.910217 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:58.910376 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:58.910916 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:58.910921 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:58.910935 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:25:58.911153 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.911164 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.912352 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.912358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.914025 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:58.914044 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:58.914921 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:58.914928 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:58.915134 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:58.915221 (+ 87us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:25:58.916382 (+ 1161us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:25:58.916409 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 00:25:58.916534 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 00:25:58.916715 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225}
I20260811 00:25:58.926460 4890 raft_consensus.cc:1261] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162 [term 1 FOLLOWER]: Refusing update from remote peer ae35798ed19f48dca0783ea23994c706: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:25:58.926435 4723 raft_consensus.cc:1261] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992 [term 1 FOLLOWER]: Refusing update from remote peer ae35798ed19f48dca0783ea23994c706: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:25:58.927464 5144 consensus_queue.cc:1035] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:25:58.927909 5165 consensus_queue.cc:1035] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:25:58.951851 5142 tablet_replica.cc:439] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992: TxnStatusTablet state changed. Reason: New leader ae35798ed19f48dca0783ea23994c706. Latest consensus state: current_term: 1 leader_uuid: "ae35798ed19f48dca0783ea23994c706" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } }
I20260811 00:25:58.952229 5142 tablet_replica.cc:442] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:25:58.949180 5143 tablet_replica.cc:439] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162: TxnStatusTablet state changed. Reason: New leader ae35798ed19f48dca0783ea23994c706. Latest consensus state: current_term: 1 leader_uuid: "ae35798ed19f48dca0783ea23994c706" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } }
I20260811 00:25:58.952373 5143 tablet_replica.cc:442] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:25:58.952391 5142 consensus_queue.cc:1035] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:25:58.952715 5170 mvcc.cc:205] Tried to move back new op lower bound from 7317126999751684096 to 7317126999408488448. Current Snapshot: MvccSnapshot[applied={T|T < 7317126999751684096}]
I20260811 00:25:58.956971 5142 consensus_queue.cc:1035] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:25:58.968096 5143 tablet_replica.cc:439] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ae35798ed19f48dca0783ea23994c706" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } }
I20260811 00:25:58.968437 5143 tablet_replica.cc:442] T 5a7e81fa8b7643bf887c6007c14c045d P 0ef8379679be4d2fac916a81e4ea7162: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:25:58.986320 5144 tablet_replica.cc:439] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ae35798ed19f48dca0783ea23994c706" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } }
I20260811 00:25:58.986696 5144 tablet_replica.cc:442] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:25:58.994134 5165 tablet_replica.cc:439] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ae35798ed19f48dca0783ea23994c706" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } }
I20260811 00:25:58.994477 5165 tablet_replica.cc:442] T 5a7e81fa8b7643bf887c6007c14c045d P ae35798ed19f48dca0783ea23994c706: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:25:58.996261 5172 mvcc.cc:205] Tried to move back new op lower bound from 7317126999751684096 to 7317126999408488448. Current Snapshot: MvccSnapshot[applied={T|T < 7317126999751684096}]
I20260811 00:25:59.001394 5142 tablet_replica.cc:439] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ae35798ed19f48dca0783ea23994c706" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } } }
I20260811 00:25:59.001744 5142 tablet_replica.cc:442] T 5a7e81fa8b7643bf887c6007c14c045d P 9e60fd992a124def95964b72fe153992: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:25:59.020802 5163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:59.003288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38728 (local address 127.2.120.1:40929)
0811 00:25:59.003523 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:59.003532 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:59.013957 (+ 10425us) server_negotiation.cc:407] Connection header received
0811 00:25:59.014180 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:59.014185 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:59.014292 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:59.014440 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:59.014959 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:59.015814 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:59.018008 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:59.018517 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:59.018646 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:59.019189 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:59.019879 (+ 690us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:59.020015 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:59.020266 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 00:25:59.020358 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 00:25:59.041738 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:59.034956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:42487 (local address 127.2.120.3:38583)
0811 00:25:59.035200 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:59.035210 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:59.035815 (+ 605us) server_negotiation.cc:407] Connection header received
0811 00:25:59.036004 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:59.036009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:59.036104 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:59.036190 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:59.036821 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:59.037591 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:59.039699 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:59.040215 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:59.040319 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:59.040988 (+ 669us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:59.041116 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:59.041198 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:59.041371 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 00:25:59.041451 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:25:59.042012 5189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:59.034908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:38583 (local address 127.2.120.1:42487)
0811 00:25:59.035658 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:59.035681 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:59.035858 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:59.036240 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:59.036246 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:59.036261 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:25:59.036635 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:59.036649 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:59.037731 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:59.037737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:59.039531 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:59.039551 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:59.040397 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:59.040405 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:59.040623 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:59.040718 (+ 95us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:25:59.041229 (+ 511us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:25:59.041249 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 00:25:59.041339 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 00:25:59.041502 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":383,"threads_started":1}
I20260811 00:25:59.088435 5163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:59.080693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:56111 (local address 127.2.120.1:40929)
0811 00:25:59.080872 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:59.080877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:59.081548 (+ 671us) server_negotiation.cc:407] Connection header received
0811 00:25:59.081815 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:59.081827 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:59.081910 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:59.082046 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:59.082781 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:59.083685 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:59.086039 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:59.086626 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:59.086736 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:59.087428 (+ 692us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:59.087580 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:59.087702 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:59.087959 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 00:25:59.088071 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 00:25:59.089226 5192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:59.080735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:40929 (local address 127.2.120.3:56111)
0811 00:25:59.081429 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:59.081445 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:59.081632 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:59.082089 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:59.082095 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:59.082108 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:25:59.082562 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:59.082576 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:59.083831 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:59.083837 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:59.085848 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:59.085874 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:59.086830 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:59.086837 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:59.087042 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:59.087132 (+ 90us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:25:59.087776 (+ 644us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:25:59.087790 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 00:25:59.088589 (+ 799us) client_negotiation.cc:240] Negotiation successful
0811 00:25:59.088761 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":384,"threads_started":1}
I20260811 00:25:59.138311 4538 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:25:59.138674 4538 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:25:59.144145 4538 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a: Sending DeleteTablet for 3 replicas of tablet de9f5babac9440c487193cb0b9ae6405
I20260811 00:25:59.146040 4697 tablet_service.cc:1514] Processing DeleteTablet for tablet de9f5babac9440c487193cb0b9ae6405 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:25:59 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:38716
I20260811 00:25:59.146312 4866 tablet_service.cc:1514] Processing DeleteTablet for tablet de9f5babac9440c487193cb0b9ae6405 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:25:59 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:41748
I20260811 00:25:59.146890 5029 tablet_service.cc:1514] Processing DeleteTablet for tablet de9f5babac9440c487193cb0b9ae6405 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:25:59 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:40020
I20260811 00:25:59.149778 4536 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:25:59.150674 4536 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:25:59.161162 5028 tablet_service.cc:1467] Processing CreateTablet for tablet 2c534444bd644796aa80f8a00efdd7e7 (DEFAULT_TABLE table=test-table [id=7dfa7b26618146859cc1b320aff64e37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:25:59.161530 4865 tablet_service.cc:1467] Processing CreateTablet for tablet 2c534444bd644796aa80f8a00efdd7e7 (DEFAULT_TABLE table=test-table [id=7dfa7b26618146859cc1b320aff64e37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:25:59.161944 5028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c534444bd644796aa80f8a00efdd7e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:59.162061 4865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c534444bd644796aa80f8a00efdd7e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:59.165113 4696 tablet_service.cc:1467] Processing CreateTablet for tablet 2c534444bd644796aa80f8a00efdd7e7 (DEFAULT_TABLE table=test-table [id=7dfa7b26618146859cc1b320aff64e37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:25:59.165689 4696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c534444bd644796aa80f8a00efdd7e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:25:59.176254 5197 tablet_replica.cc:330] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706: stopping tablet replica
I20260811 00:25:59.176810 5197 raft_consensus.cc:2229] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:25:59.177191 5197 raft_consensus.cc:2258] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:25:59.179126 5196 tablet_replica.cc:330] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162: stopping tablet replica
I20260811 00:25:59.179931 5196 raft_consensus.cc:2229] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:25:59.180549 5196 raft_consensus.cc:2258] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:25:59.182650 5197 ts_tablet_manager.cc:1905] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:25:59.185034 5196 ts_tablet_manager.cc:1905] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:25:59.192982 5195 tablet_replica.cc:330] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992: stopping tablet replica
I20260811 00:25:59.193065 5200 tablet_bootstrap.cc:492] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992: Bootstrap starting.
I20260811 00:25:59.193593 5195 raft_consensus.cc:2229] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:25:59.194445 5195 raft_consensus.cc:2258] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:25:59.195464 5200 tablet_bootstrap.cc:654] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:59.196132 5198 tablet_bootstrap.cc:492] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162: Bootstrap starting.
I20260811 00:25:59.196367 5196 ts_tablet_manager.cc:1918] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:25:59.196684 5196 log.cc:1196] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/de9f5babac9440c487193cb0b9ae6405
I20260811 00:25:59.196931 5195 ts_tablet_manager.cc:1905] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:25:59.199275 5196 ts_tablet_manager.cc:1939] T de9f5babac9440c487193cb0b9ae6405 P 0ef8379679be4d2fac916a81e4ea7162: Deleting consensus metadata
I20260811 00:25:59.201157 5199 tablet_bootstrap.cc:492] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706: Bootstrap starting.
I20260811 00:25:59.198292 5198 tablet_bootstrap.cc:654] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:59.201632 4521 catalog_manager.cc:4841] TS 0ef8379679be4d2fac916a81e4ea7162 (127.2.120.2:33923): tablet de9f5babac9440c487193cb0b9ae6405 (table test-table [id=35d15ab82a5b4e0ebab18fbfaeb074e2]) successfully deleted
I20260811 00:25:59.203562 5199 tablet_bootstrap.cc:654] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706: Neither blocks nor log segments found. Creating new log.
I20260811 00:25:59.210192 5197 ts_tablet_manager.cc:1918] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:25:59.210516 5197 log.cc:1196] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/de9f5babac9440c487193cb0b9ae6405
I20260811 00:25:59.211236 5197 ts_tablet_manager.cc:1939] T de9f5babac9440c487193cb0b9ae6405 P ae35798ed19f48dca0783ea23994c706: Deleting consensus metadata
I20260811 00:25:59.211937 5195 ts_tablet_manager.cc:1918] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:25:59.212209 5195 log.cc:1196] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/de9f5babac9440c487193cb0b9ae6405
I20260811 00:25:59.212939 5195 ts_tablet_manager.cc:1939] T de9f5babac9440c487193cb0b9ae6405 P 9e60fd992a124def95964b72fe153992: Deleting consensus metadata
I20260811 00:25:59.214011 5198 tablet_bootstrap.cc:492] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162: No bootstrap required, opened a new log
I20260811 00:25:59.214213 5198 ts_tablet_manager.cc:1397] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.000s
I20260811 00:25:59.215512 4522 catalog_manager.cc:4841] TS 9e60fd992a124def95964b72fe153992 (127.2.120.1:40929): tablet de9f5babac9440c487193cb0b9ae6405 (table test-table [id=35d15ab82a5b4e0ebab18fbfaeb074e2]) successfully deleted
I20260811 00:25:59.216332 5198 raft_consensus.cc:348] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:25:59.216737 5198 raft_consensus.cc:374] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:59.216854 5198 raft_consensus.cc:729] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ef8379679be4d2fac916a81e4ea7162, State: Initialized, Role: FOLLOWER
I20260811 00:25:59.217216 5198 consensus_queue.cc:260] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:25:59.219522 5198 ts_tablet_manager.cc:1428] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260811 00:25:59.219758 5199 tablet_bootstrap.cc:492] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706: No bootstrap required, opened a new log
I20260811 00:25:59.219942 5199 ts_tablet_manager.cc:1397] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.000s
I20260811 00:25:59.220889 4521 catalog_manager.cc:4841] TS ae35798ed19f48dca0783ea23994c706 (127.2.120.3:38583): tablet de9f5babac9440c487193cb0b9ae6405 (table test-table [id=35d15ab82a5b4e0ebab18fbfaeb074e2]) successfully deleted
I20260811 00:25:59.221498 5199 raft_consensus.cc:348] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:25:59.221736 5199 raft_consensus.cc:374] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:59.221838 5199 raft_consensus.cc:729] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae35798ed19f48dca0783ea23994c706, State: Initialized, Role: FOLLOWER
I20260811 00:25:59.222124 5199 consensus_queue.cc:260] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:25:59.222816 5199 ts_tablet_manager.cc:1428] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 00:25:59.230422 5200 tablet_bootstrap.cc:492] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992: No bootstrap required, opened a new log
I20260811 00:25:59.230657 5200 ts_tablet_manager.cc:1397] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992: Time spent bootstrapping tablet: real 0.038s user 0.003s sys 0.010s
I20260811 00:25:59.231690 5200 raft_consensus.cc:348] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:25:59.232066 5200 raft_consensus.cc:374] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:25:59.232172 5200 raft_consensus.cc:729] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e60fd992a124def95964b72fe153992, State: Initialized, Role: FOLLOWER
I20260811 00:25:59.232445 5200 consensus_queue.cc:260] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:25:59.233289 5200 ts_tablet_manager.cc:1428] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 00:25:59.417403 5165 raft_consensus.cc:482] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:25:59.417779 5165 raft_consensus.cc:504] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:25:59.418901 5165 leader_election.cc:288] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e60fd992a124def95964b72fe153992 (127.2.120.1:40929), 0ef8379679be4d2fac916a81e4ea7162 (127.2.120.2:33923)
I20260811 00:25:59.419518 4722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c534444bd644796aa80f8a00efdd7e7" candidate_uuid: "ae35798ed19f48dca0783ea23994c706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e60fd992a124def95964b72fe153992" is_pre_election: true
I20260811 00:25:59.419605 4889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c534444bd644796aa80f8a00efdd7e7" candidate_uuid: "ae35798ed19f48dca0783ea23994c706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef8379679be4d2fac916a81e4ea7162" is_pre_election: true
I20260811 00:25:59.419885 4889 raft_consensus.cc:2454] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae35798ed19f48dca0783ea23994c706 in term 0.
I20260811 00:25:59.419890 4722 raft_consensus.cc:2454] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae35798ed19f48dca0783ea23994c706 in term 0.
I20260811 00:25:59.420459 4973 leader_election.cc:302] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e60fd992a124def95964b72fe153992, ae35798ed19f48dca0783ea23994c706; no voters:
I20260811 00:25:59.420872 5165 raft_consensus.cc:2781] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:25:59.421011 5165 raft_consensus.cc:482] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:25:59.421126 5165 raft_consensus.cc:3037] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:59.424469 5165 raft_consensus.cc:504] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:25:59.425420 5165 leader_election.cc:288] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [CANDIDATE]: Term 1 election: Requested vote from peers 9e60fd992a124def95964b72fe153992 (127.2.120.1:40929), 0ef8379679be4d2fac916a81e4ea7162 (127.2.120.2:33923)
I20260811 00:25:59.426214 4722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c534444bd644796aa80f8a00efdd7e7" candidate_uuid: "ae35798ed19f48dca0783ea23994c706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e60fd992a124def95964b72fe153992"
I20260811 00:25:59.426198 4889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c534444bd644796aa80f8a00efdd7e7" candidate_uuid: "ae35798ed19f48dca0783ea23994c706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef8379679be4d2fac916a81e4ea7162"
I20260811 00:25:59.426456 4889 raft_consensus.cc:3037] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:59.426506 4722 raft_consensus.cc:3037] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:25:59.429845 4722 raft_consensus.cc:2454] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae35798ed19f48dca0783ea23994c706 in term 1.
I20260811 00:25:59.430233 4889 raft_consensus.cc:2454] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae35798ed19f48dca0783ea23994c706 in term 1.
I20260811 00:25:59.430555 4973 leader_election.cc:302] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e60fd992a124def95964b72fe153992, ae35798ed19f48dca0783ea23994c706; no voters:
I20260811 00:25:59.430953 5165 raft_consensus.cc:2781] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:25:59.431214 5165 raft_consensus.cc:686] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [term 1 LEADER]: Becoming Leader. State: Replica: ae35798ed19f48dca0783ea23994c706, State: Running, Role: LEADER
I20260811 00:25:59.431597 5165 consensus_queue.cc:237] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:25:59.435362 4538 catalog_manager.cc:5495] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae35798ed19f48dca0783ea23994c706 (127.2.120.3). New cstate: current_term: 1 leader_uuid: "ae35798ed19f48dca0783ea23994c706" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } health_report { overall_health: HEALTHY } } }
I20260811 00:25:59.492309 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:59.484739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40044 (local address 127.2.120.3:38583)
0811 00:25:59.484963 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:59.484970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:59.484991 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:25:59.485071 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:59.485076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:59.485176 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:59.485274 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:59.485915 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:59.487059 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:59.489263 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:59.490047 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:59.490194 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:59.490957 (+ 763us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:59.491132 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:59.491232 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:59.491853 (+ 621us) server_negotiation.cc:299] Negotiation successful
0811 00:25:59.491959 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 00:25:59.510149 5192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:59.503107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:38583 (local address 127.2.120.3:36865)
0811 00:25:59.503404 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 00:25:59.503423 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:25:59.503551 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:25:59.503972 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:25:59.503977 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:25:59.503997 (+ 20us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:25:59.504180 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:59.504188 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:59.506031 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:59.506036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:59.507214 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:25:59.507229 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:25:59.508753 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:25:59.508762 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:25:59.508946 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:59.509015 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:25:59.509623 (+ 608us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:25:59.509638 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 00:25:59.509729 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 00:25:59.509867 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179}
I20260811 00:25:59.510232 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:25:59.503055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:36865 (local address 127.2.120.3:38583)
0811 00:25:59.503238 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 00:25:59.503249 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:25:59.503669 (+ 420us) server_negotiation.cc:407] Connection header received
0811 00:25:59.503724 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:25:59.503729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:25:59.503804 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:25:59.503905 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:25:59.504356 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:59.505837 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:59.507391 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:25:59.508153 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:25:59.508311 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:25:59.509288 (+ 977us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:25:59.509451 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:25:59.509576 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 00:25:59.509786 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 00:25:59.509878 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 00:25:59.521603 4722 raft_consensus.cc:1261] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992 [term 1 FOLLOWER]: Refusing update from remote peer ae35798ed19f48dca0783ea23994c706: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:25:59.521574 4889 raft_consensus.cc:1261] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162 [term 1 FOLLOWER]: Refusing update from remote peer ae35798ed19f48dca0783ea23994c706: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:25:59.522341 5181 consensus_queue.cc:1035] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:25:59.522720 5165 consensus_queue.cc:1035] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:25:59.535151 5171 mvcc.cc:205] Tried to move back new op lower bound from 7317127002194198528 to 7317127001835278336. Current Snapshot: MvccSnapshot[applied={T|T < 7317127002194198528}]
W20260811 00:25:59.553133 5170 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 00:26:01.188038 4536 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:01.188306 4536 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:01.194353 4536 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a: Sending DeleteTablet for 3 replicas of tablet 2c534444bd644796aa80f8a00efdd7e7
I20260811 00:26:01.195365 4697 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c534444bd644796aa80f8a00efdd7e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:01 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:38716
I20260811 00:26:01.195621 5028 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c534444bd644796aa80f8a00efdd7e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:01 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:40020
I20260811 00:26:01.195890 4865 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c534444bd644796aa80f8a00efdd7e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:01 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:41748
I20260811 00:26:01.208351 5235 tablet_replica.cc:330] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992: stopping tablet replica
I20260811 00:26:01.208626 5237 tablet_replica.cc:330] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162: stopping tablet replica
I20260811 00:26:01.208907 5235 raft_consensus.cc:2229] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:01.209340 5235 raft_consensus.cc:2258] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:01.209442 5237 raft_consensus.cc:2229] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:01.210067 5237 raft_consensus.cc:2258] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:01.210753 5235 ts_tablet_manager.cc:1905] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:01.211689 5237 ts_tablet_manager.cc:1905] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:01.216677 5236 tablet_replica.cc:330] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706: stopping tablet replica
I20260811 00:26:01.217034 5236 raft_consensus.cc:2229] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:26:01.219422 5235 ts_tablet_manager.cc:1918] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 00:26:01.219609 5236 raft_consensus.cc:2258] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:01.219717 5235 log.cc:1196] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/2c534444bd644796aa80f8a00efdd7e7
I20260811 00:26:01.220384 5235 ts_tablet_manager.cc:1939] T 2c534444bd644796aa80f8a00efdd7e7 P 9e60fd992a124def95964b72fe153992: Deleting consensus metadata
I20260811 00:26:01.220890 5236 ts_tablet_manager.cc:1905] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:01.221905 4522 catalog_manager.cc:4841] TS 9e60fd992a124def95964b72fe153992 (127.2.120.1:40929): tablet 2c534444bd644796aa80f8a00efdd7e7 (table test-table [id=7dfa7b26618146859cc1b320aff64e37]) successfully deleted
I20260811 00:26:01.225848 5237 ts_tablet_manager.cc:1918] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 00:26:01.226118 5237 log.cc:1196] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/2c534444bd644796aa80f8a00efdd7e7
I20260811 00:26:01.226687 5237 ts_tablet_manager.cc:1939] T 2c534444bd644796aa80f8a00efdd7e7 P 0ef8379679be4d2fac916a81e4ea7162: Deleting consensus metadata
I20260811 00:26:01.228086 4521 catalog_manager.cc:4841] TS 0ef8379679be4d2fac916a81e4ea7162 (127.2.120.2:33923): tablet 2c534444bd644796aa80f8a00efdd7e7 (table test-table [id=7dfa7b26618146859cc1b320aff64e37]) successfully deleted
I20260811 00:26:01.228746 5236 ts_tablet_manager.cc:1918] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 00:26:01.228924 5236 log.cc:1196] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/2c534444bd644796aa80f8a00efdd7e7
I20260811 00:26:01.229362 5236 ts_tablet_manager.cc:1939] T 2c534444bd644796aa80f8a00efdd7e7 P ae35798ed19f48dca0783ea23994c706: Deleting consensus metadata
I20260811 00:26:01.230545 4521 catalog_manager.cc:4841] TS ae35798ed19f48dca0783ea23994c706 (127.2.120.3:38583): tablet 2c534444bd644796aa80f8a00efdd7e7 (table test-table [id=7dfa7b26618146859cc1b320aff64e37]) successfully deleted
Aug 11 00:26:01 dist-test-slave-fd7l krb5kdc[4485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407961, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 00:26:01 dist-test-slave-fd7l krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407961, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:01.283429 5246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:01.255459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59886 (local address 127.2.120.62:34195)
0811 00:26:01.256173 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:01.256178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:01.256202 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:26:01.256413 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:01.256417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:01.256498 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:01.256651 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:01.259169 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:01.260091 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:01.262517 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:01.262983 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:01.263112 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:01.275563 (+ 12451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:01.275573 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:01.275577 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:01.275678 (+ 101us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:01.279982 (+ 4304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:01.281007 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:01.281012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:01.281015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:01.281200 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:01.281736 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:01.281741 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:01.281743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:01.282195 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:01.282464 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:01.282776 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 00:26:01.283015 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":401,"threads_started":1}
I20260811 00:26:01.288071 4536 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:26:01.288792 4536 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:26:01.297343 4697 tablet_service.cc:1467] Processing CreateTablet for tablet decbacba9686427992d5672c14649e66 (DEFAULT_TABLE table=test-table [id=3e6ffe9d63d54c97aa60ac810c70949d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:01.297483 5028 tablet_service.cc:1467] Processing CreateTablet for tablet decbacba9686427992d5672c14649e66 (DEFAULT_TABLE table=test-table [id=3e6ffe9d63d54c97aa60ac810c70949d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:01.297428 4865 tablet_service.cc:1467] Processing CreateTablet for tablet decbacba9686427992d5672c14649e66 (DEFAULT_TABLE table=test-table [id=3e6ffe9d63d54c97aa60ac810c70949d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:01.297935 4697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet decbacba9686427992d5672c14649e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:01.297969 5028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet decbacba9686427992d5672c14649e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:01.297971 4865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet decbacba9686427992d5672c14649e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:01.305485 5247 tablet_bootstrap.cc:492] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162: Bootstrap starting.
I20260811 00:26:01.306288 5249 tablet_bootstrap.cc:492] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992: Bootstrap starting.
I20260811 00:26:01.308053 5247 tablet_bootstrap.cc:654] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:01.309283 5248 tablet_bootstrap.cc:492] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706: Bootstrap starting.
I20260811 00:26:01.308286 5249 tablet_bootstrap.cc:654] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:01.311702 5248 tablet_bootstrap.cc:654] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:01.312278 5247 tablet_bootstrap.cc:492] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162: No bootstrap required, opened a new log
I20260811 00:26:01.312314 5249 tablet_bootstrap.cc:492] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992: No bootstrap required, opened a new log
I20260811 00:26:01.312489 5249 ts_tablet_manager.cc:1397] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 00:26:01.312493 5247 ts_tablet_manager.cc:1397] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 00:26:01.313557 5247 raft_consensus.cc:348] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:26:01.313843 5247 raft_consensus.cc:374] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:01.313964 5247 raft_consensus.cc:729] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ef8379679be4d2fac916a81e4ea7162, State: Initialized, Role: FOLLOWER
I20260811 00:26:01.313656 5249 raft_consensus.cc:348] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:26:01.314376 5249 raft_consensus.cc:374] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:01.314304 5247 consensus_queue.cc:260] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:26:01.314528 5249 raft_consensus.cc:729] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e60fd992a124def95964b72fe153992, State: Initialized, Role: FOLLOWER
I20260811 00:26:01.314884 5249 consensus_queue.cc:260] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:26:01.315567 5247 ts_tablet_manager.cc:1428] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 00:26:01.321273 5249 ts_tablet_manager.cc:1428] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992: Time spent starting tablet: real 0.009s user 0.001s sys 0.008s
I20260811 00:26:01.321318 5248 tablet_bootstrap.cc:492] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706: No bootstrap required, opened a new log
I20260811 00:26:01.321507 5248 ts_tablet_manager.cc:1397] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.006s
I20260811 00:26:01.322415 5248 raft_consensus.cc:348] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:26:01.322701 5248 raft_consensus.cc:374] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:01.322804 5248 raft_consensus.cc:729] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae35798ed19f48dca0783ea23994c706, State: Initialized, Role: FOLLOWER
I20260811 00:26:01.323101 5248 consensus_queue.cc:260] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:26:01.323782 5248 ts_tablet_manager.cc:1428] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 00:26:01.418011 5144 raft_consensus.cc:482] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:26:01.418407 5144 raft_consensus.cc:504] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:26:01.419263 5144 leader_election.cc:288] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e60fd992a124def95964b72fe153992 (127.2.120.1:40929), 0ef8379679be4d2fac916a81e4ea7162 (127.2.120.2:33923)
I20260811 00:26:01.419934 4723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "decbacba9686427992d5672c14649e66" candidate_uuid: "ae35798ed19f48dca0783ea23994c706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e60fd992a124def95964b72fe153992" is_pre_election: true
I20260811 00:26:01.419951 4890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "decbacba9686427992d5672c14649e66" candidate_uuid: "ae35798ed19f48dca0783ea23994c706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef8379679be4d2fac916a81e4ea7162" is_pre_election: true
I20260811 00:26:01.420212 4723 raft_consensus.cc:2454] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae35798ed19f48dca0783ea23994c706 in term 0.
I20260811 00:26:01.420321 4890 raft_consensus.cc:2454] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae35798ed19f48dca0783ea23994c706 in term 0.
I20260811 00:26:01.420877 4973 leader_election.cc:302] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e60fd992a124def95964b72fe153992, ae35798ed19f48dca0783ea23994c706; no voters:
I20260811 00:26:01.421190 5144 raft_consensus.cc:2781] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:26:01.421322 5144 raft_consensus.cc:482] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:26:01.421398 5144 raft_consensus.cc:3037] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:01.423534 5144 raft_consensus.cc:504] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:26:01.424167 5144 leader_election.cc:288] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [CANDIDATE]: Term 1 election: Requested vote from peers 9e60fd992a124def95964b72fe153992 (127.2.120.1:40929), 0ef8379679be4d2fac916a81e4ea7162 (127.2.120.2:33923)
I20260811 00:26:01.424892 4723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "decbacba9686427992d5672c14649e66" candidate_uuid: "ae35798ed19f48dca0783ea23994c706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e60fd992a124def95964b72fe153992"
I20260811 00:26:01.424885 4890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "decbacba9686427992d5672c14649e66" candidate_uuid: "ae35798ed19f48dca0783ea23994c706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef8379679be4d2fac916a81e4ea7162"
I20260811 00:26:01.425151 4723 raft_consensus.cc:3037] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:01.425179 4890 raft_consensus.cc:3037] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:01.428463 4723 raft_consensus.cc:2454] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae35798ed19f48dca0783ea23994c706 in term 1.
I20260811 00:26:01.428454 4890 raft_consensus.cc:2454] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae35798ed19f48dca0783ea23994c706 in term 1.
I20260811 00:26:01.429176 4972 leader_election.cc:302] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ef8379679be4d2fac916a81e4ea7162, ae35798ed19f48dca0783ea23994c706; no voters:
I20260811 00:26:01.429582 5144 raft_consensus.cc:2781] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:01.429759 5144 raft_consensus.cc:686] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [term 1 LEADER]: Becoming Leader. State: Replica: ae35798ed19f48dca0783ea23994c706, State: Running, Role: LEADER
I20260811 00:26:01.430133 5144 consensus_queue.cc:237] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:26:01.433280 4539 catalog_manager.cc:5495] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae35798ed19f48dca0783ea23994c706 (127.2.120.3). New cstate: current_term: 1 leader_uuid: "ae35798ed19f48dca0783ea23994c706" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } health_report { overall_health: HEALTHY } } }
I20260811 00:26:01.491451 5258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:01.482615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40060 (local address 127.2.120.3:38583)
0811 00:26:01.485380 (+ 2765us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:01.485385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:01.485417 (+ 32us) server_negotiation.cc:407] Connection header received
0811 00:26:01.485510 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:01.485514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:01.485604 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:01.485894 (+ 290us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:26:01.486402 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:01.487233 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:01.489312 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:01.489816 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:01.490052 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:01.490574 (+ 522us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:26:01.490707 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:26:01.490824 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:01.491001 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 00:26:01.491116 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2601,"thread_start_us":2402,"threads_started":1}
I20260811 00:26:01.502348 4890 raft_consensus.cc:1261] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162 [term 1 FOLLOWER]: Refusing update from remote peer ae35798ed19f48dca0783ea23994c706: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:01.502511 4723 raft_consensus.cc:1261] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992 [term 1 FOLLOWER]: Refusing update from remote peer ae35798ed19f48dca0783ea23994c706: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:01.503036 5144 consensus_queue.cc:1035] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:01.503399 5220 consensus_queue.cc:1035] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:01.563411 4536 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:01.563675 4536 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:01.568702 4536 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a: Sending DeleteTablet for 3 replicas of tablet decbacba9686427992d5672c14649e66
I20260811 00:26:01.569854 4865 tablet_service.cc:1514] Processing DeleteTablet for tablet decbacba9686427992d5672c14649e66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:01 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:41748
I20260811 00:26:01.569932 5028 tablet_service.cc:1514] Processing DeleteTablet for tablet decbacba9686427992d5672c14649e66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:01 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:40020
I20260811 00:26:01.569893 4697 tablet_service.cc:1514] Processing DeleteTablet for tablet decbacba9686427992d5672c14649e66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:01 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:38716
I20260811 00:26:01.570231 5237 tablet_replica.cc:330] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162: stopping tablet replica
I20260811 00:26:01.570230 5236 tablet_replica.cc:330] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706: stopping tablet replica
I20260811 00:26:01.570326 5235 tablet_replica.cc:330] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992: stopping tablet replica
I20260811 00:26:01.570447 5237 raft_consensus.cc:2229] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:01.570448 5236 raft_consensus.cc:2229] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:26:01.570637 5235 raft_consensus.cc:2229] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:01.570683 5237 raft_consensus.cc:2258] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:01.570876 5235 raft_consensus.cc:2258] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:01.570860 5236 raft_consensus.cc:2258] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:01.571586 4536 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 00:26:01.572062 5236 ts_tablet_manager.cc:1905] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:01.572100 5235 ts_tablet_manager.cc:1905] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:01.572106 5237 ts_tablet_manager.cc:1905] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 00:26:01.572420 4536 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:26:01.579792 5237 ts_tablet_manager.cc:1918] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:26:01.580063 5237 log.cc:1196] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/decbacba9686427992d5672c14649e66
I20260811 00:26:01.580500 5235 ts_tablet_manager.cc:1918] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:26:01.580699 5237 ts_tablet_manager.cc:1939] T decbacba9686427992d5672c14649e66 P 0ef8379679be4d2fac916a81e4ea7162: Deleting consensus metadata
I20260811 00:26:01.580832 5235 log.cc:1196] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/decbacba9686427992d5672c14649e66
I20260811 00:26:01.581534 5235 ts_tablet_manager.cc:1939] T decbacba9686427992d5672c14649e66 P 9e60fd992a124def95964b72fe153992: Deleting consensus metadata
I20260811 00:26:01.582090 5236 ts_tablet_manager.cc:1918] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:26:01.582314 5236 log.cc:1196] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/decbacba9686427992d5672c14649e66
I20260811 00:26:01.582384 4865 tablet_service.cc:1467] Processing CreateTablet for tablet 7daae3562d644a73a07ef92bb50619a3 (DEFAULT_TABLE table=test-table [id=768ee5744d364a7cb85daf7157332613]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:01.582844 5236 ts_tablet_manager.cc:1939] T decbacba9686427992d5672c14649e66 P ae35798ed19f48dca0783ea23994c706: Deleting consensus metadata
I20260811 00:26:01.582938 4865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7daae3562d644a73a07ef92bb50619a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:01.582871 4697 tablet_service.cc:1467] Processing CreateTablet for tablet 7daae3562d644a73a07ef92bb50619a3 (DEFAULT_TABLE table=test-table [id=768ee5744d364a7cb85daf7157332613]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:01.583247 4522 catalog_manager.cc:4841] TS 9e60fd992a124def95964b72fe153992 (127.2.120.1:40929): tablet decbacba9686427992d5672c14649e66 (table test-table [id=3e6ffe9d63d54c97aa60ac810c70949d]) successfully deleted
I20260811 00:26:01.583333 4697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7daae3562d644a73a07ef92bb50619a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:01.584156 4521 catalog_manager.cc:4841] TS ae35798ed19f48dca0783ea23994c706 (127.2.120.3:38583): tablet decbacba9686427992d5672c14649e66 (table test-table [id=3e6ffe9d63d54c97aa60ac810c70949d]) successfully deleted
I20260811 00:26:01.585120 4521 catalog_manager.cc:4841] TS 0ef8379679be4d2fac916a81e4ea7162 (127.2.120.2:33923): tablet decbacba9686427992d5672c14649e66 (table test-table [id=3e6ffe9d63d54c97aa60ac810c70949d]) successfully deleted
I20260811 00:26:01.585739 5028 tablet_service.cc:1467] Processing CreateTablet for tablet 7daae3562d644a73a07ef92bb50619a3 (DEFAULT_TABLE table=test-table [id=768ee5744d364a7cb85daf7157332613]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:01.586193 5028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7daae3562d644a73a07ef92bb50619a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:01.590903 5247 tablet_bootstrap.cc:492] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162: Bootstrap starting.
I20260811 00:26:01.592131 5249 tablet_bootstrap.cc:492] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992: Bootstrap starting.
I20260811 00:26:01.593190 5248 tablet_bootstrap.cc:492] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706: Bootstrap starting.
I20260811 00:26:01.593513 5247 tablet_bootstrap.cc:654] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:01.594179 5249 tablet_bootstrap.cc:654] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:01.595160 5248 tablet_bootstrap.cc:654] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:01.596297 5247 tablet_bootstrap.cc:492] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162: No bootstrap required, opened a new log
I20260811 00:26:01.596531 5247 ts_tablet_manager.cc:1397] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260811 00:26:01.597429 5247 raft_consensus.cc:348] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:26:01.597683 5248 tablet_bootstrap.cc:492] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706: No bootstrap required, opened a new log
I20260811 00:26:01.597764 5247 raft_consensus.cc:374] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:01.597872 5247 raft_consensus.cc:729] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ef8379679be4d2fac916a81e4ea7162, State: Initialized, Role: FOLLOWER
I20260811 00:26:01.597910 5248 ts_tablet_manager.cc:1397] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260811 00:26:01.598201 5247 consensus_queue.cc:260] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:26:01.598537 5249 tablet_bootstrap.cc:492] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992: No bootstrap required, opened a new log
I20260811 00:26:01.598714 5249 ts_tablet_manager.cc:1397] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 00:26:01.598779 5248 raft_consensus.cc:348] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:26:01.598953 5247 ts_tablet_manager.cc:1428] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 00:26:01.599028 5248 raft_consensus.cc:374] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:01.599112 5248 raft_consensus.cc:729] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae35798ed19f48dca0783ea23994c706, State: Initialized, Role: FOLLOWER
I20260811 00:26:01.599400 5248 consensus_queue.cc:260] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:26:01.599658 5249 raft_consensus.cc:348] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:26:01.599887 5249 raft_consensus.cc:374] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:01.599965 5248 ts_tablet_manager.cc:1428] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 00:26:01.599972 5249 raft_consensus.cc:729] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e60fd992a124def95964b72fe153992, State: Initialized, Role: FOLLOWER
I20260811 00:26:01.600245 5249 consensus_queue.cc:260] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:26:01.600950 5249 ts_tablet_manager.cc:1428] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 00:26:01.673209 5144 raft_consensus.cc:482] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:26:01.673569 5144 raft_consensus.cc:504] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:26:01.674535 5144 leader_election.cc:288] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e60fd992a124def95964b72fe153992 (127.2.120.1:40929), 0ef8379679be4d2fac916a81e4ea7162 (127.2.120.2:33923)
I20260811 00:26:01.675706 4889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7daae3562d644a73a07ef92bb50619a3" candidate_uuid: "ae35798ed19f48dca0783ea23994c706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef8379679be4d2fac916a81e4ea7162" is_pre_election: true
I20260811 00:26:01.675806 4723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7daae3562d644a73a07ef92bb50619a3" candidate_uuid: "ae35798ed19f48dca0783ea23994c706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e60fd992a124def95964b72fe153992" is_pre_election: true
I20260811 00:26:01.676093 4889 raft_consensus.cc:2454] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae35798ed19f48dca0783ea23994c706 in term 0.
I20260811 00:26:01.676156 4723 raft_consensus.cc:2454] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae35798ed19f48dca0783ea23994c706 in term 0.
I20260811 00:26:01.676757 4973 leader_election.cc:302] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0ef8379679be4d2fac916a81e4ea7162, 9e60fd992a124def95964b72fe153992, ae35798ed19f48dca0783ea23994c706; no voters:
I20260811 00:26:01.677326 5144 raft_consensus.cc:2781] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:26:01.677517 5144 raft_consensus.cc:482] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:26:01.677701 5144 raft_consensus.cc:3037] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:01.680097 5144 raft_consensus.cc:504] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:26:01.681020 5144 leader_election.cc:288] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [CANDIDATE]: Term 1 election: Requested vote from peers 9e60fd992a124def95964b72fe153992 (127.2.120.1:40929), 0ef8379679be4d2fac916a81e4ea7162 (127.2.120.2:33923)
I20260811 00:26:01.681800 4723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7daae3562d644a73a07ef92bb50619a3" candidate_uuid: "ae35798ed19f48dca0783ea23994c706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e60fd992a124def95964b72fe153992"
I20260811 00:26:01.681867 4889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7daae3562d644a73a07ef92bb50619a3" candidate_uuid: "ae35798ed19f48dca0783ea23994c706" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ef8379679be4d2fac916a81e4ea7162"
I20260811 00:26:01.682113 4889 raft_consensus.cc:3037] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:01.682113 4723 raft_consensus.cc:3037] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:01.685879 4723 raft_consensus.cc:2454] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae35798ed19f48dca0783ea23994c706 in term 1.
I20260811 00:26:01.685875 4889 raft_consensus.cc:2454] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae35798ed19f48dca0783ea23994c706 in term 1.
I20260811 00:26:01.686749 4973 leader_election.cc:302] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0ef8379679be4d2fac916a81e4ea7162, 9e60fd992a124def95964b72fe153992, ae35798ed19f48dca0783ea23994c706; no voters:
I20260811 00:26:01.687182 5144 raft_consensus.cc:2781] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:01.687383 5144 raft_consensus.cc:686] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [term 1 LEADER]: Becoming Leader. State: Replica: ae35798ed19f48dca0783ea23994c706, State: Running, Role: LEADER
I20260811 00:26:01.687721 5144 consensus_queue.cc:237] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } }
I20260811 00:26:01.691396 4536 catalog_manager.cc:5495] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae35798ed19f48dca0783ea23994c706 (127.2.120.3). New cstate: current_term: 1 leader_uuid: "ae35798ed19f48dca0783ea23994c706" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae35798ed19f48dca0783ea23994c706" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38583 } health_report { overall_health: HEALTHY } } }
I20260811 00:26:01.722826 4889 raft_consensus.cc:1261] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162 [term 1 FOLLOWER]: Refusing update from remote peer ae35798ed19f48dca0783ea23994c706: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:01.723469 4723 raft_consensus.cc:1261] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992 [term 1 FOLLOWER]: Refusing update from remote peer ae35798ed19f48dca0783ea23994c706: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:01.723454 5214 consensus_queue.cc:1035] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ef8379679be4d2fac916a81e4ea7162" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:01.724143 5144 consensus_queue.cc:1035] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e60fd992a124def95964b72fe153992" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 40929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 00:26:01.748322 5211 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 00:26:03.363798 4536 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:03.364064 4536 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:03.369678 4536 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5385c4e94d5e46648f9f6d41cd38ed8a: Sending DeleteTablet for 3 replicas of tablet 7daae3562d644a73a07ef92bb50619a3
I20260811 00:26:03.370788 4697 tablet_service.cc:1514] Processing DeleteTablet for tablet 7daae3562d644a73a07ef92bb50619a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:03 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:38716
I20260811 00:26:03.370886 4865 tablet_service.cc:1514] Processing DeleteTablet for tablet 7daae3562d644a73a07ef92bb50619a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:03 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:41748
I20260811 00:26:03.371023 5028 tablet_service.cc:1514] Processing DeleteTablet for tablet 7daae3562d644a73a07ef92bb50619a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:03 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:40020
I20260811 00:26:03.371706 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 4617
I20260811 00:26:03.383539 5298 tablet_replica.cc:330] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706: stopping tablet replica
I20260811 00:26:03.383850 5298 raft_consensus.cc:2229] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:26:03.383908 5293 tablet_replica.cc:330] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992: stopping tablet replica
I20260811 00:26:03.384335 5293 raft_consensus.cc:2229] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:03.384398 5298 raft_consensus.cc:2258] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:03.385414 5295 tablet_replica.cc:330] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162: stopping tablet replica
I20260811 00:26:03.385650 5293 raft_consensus.cc:2258] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:03.386333 5298 ts_tablet_manager.cc:1905] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:03.386323 5295 raft_consensus.cc:2229] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:03.386608 5295 raft_consensus.cc:2258] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:03.387143 5293 ts_tablet_manager.cc:1905] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:03.387773 5295 ts_tablet_manager.cc:1905] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:03.394511 5293 ts_tablet_manager.cc:1918] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 00:26:03.394810 5293 log.cc:1196] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/7daae3562d644a73a07ef92bb50619a3
I20260811 00:26:03.394775 5298 ts_tablet_manager.cc:1918] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 00:26:03.394937 5298 log.cc:1196] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/7daae3562d644a73a07ef92bb50619a3
I20260811 00:26:03.395362 5295 ts_tablet_manager.cc:1918] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 00:26:03.395423 5293 ts_tablet_manager.cc:1939] T 7daae3562d644a73a07ef92bb50619a3 P 9e60fd992a124def95964b72fe153992: Deleting consensus metadata
I20260811 00:26:03.395552 5298 ts_tablet_manager.cc:1939] T 7daae3562d644a73a07ef92bb50619a3 P ae35798ed19f48dca0783ea23994c706: Deleting consensus metadata
I20260811 00:26:03.395605 5295 log.cc:1196] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/7daae3562d644a73a07ef92bb50619a3
I20260811 00:26:03.396196 5295 ts_tablet_manager.cc:1939] T 7daae3562d644a73a07ef92bb50619a3 P 0ef8379679be4d2fac916a81e4ea7162: Deleting consensus metadata
I20260811 00:26:03.396974 4522 catalog_manager.cc:4841] TS 9e60fd992a124def95964b72fe153992 (127.2.120.1:40929): tablet 7daae3562d644a73a07ef92bb50619a3 (table test-table [id=768ee5744d364a7cb85daf7157332613]) successfully deleted
I20260811 00:26:03.397033 4521 catalog_manager.cc:4841] TS ae35798ed19f48dca0783ea23994c706 (127.2.120.3:38583): tablet 7daae3562d644a73a07ef92bb50619a3 (table test-table [id=768ee5744d364a7cb85daf7157332613]) successfully deleted
I20260811 00:26:03.397540 4521 catalog_manager.cc:4841] TS 0ef8379679be4d2fac916a81e4ea7162 (127.2.120.2:33923): tablet 7daae3562d644a73a07ef92bb50619a3 (table test-table [id=768ee5744d364a7cb85daf7157332613]) successfully deleted
Aug 11 00:26:03 dist-test-slave-fd7l krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407961, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:26:03.407953 5301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:03.374441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50240 (local address 127.2.120.1:40929)
0811 00:26:03.387289 (+ 12848us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:03.387298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:03.387328 (+ 30us) server_negotiation.cc:407] Connection header received
0811 00:26:03.387397 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:03.387405 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:03.387499 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:03.389460 (+ 1961us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:03.389669 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:03.390896 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:03.393075 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:03.393724 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:03.397429 (+ 3705us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:03.399794 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:03.399821 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:03.399848 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:03.399920 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:03.404496 (+ 4576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:03.405490 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:03.405499 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:03.405502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:03.405667 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:03.406176 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:03.406180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:03.406182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:03.406624 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:03.406974 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:03.407294 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 00:26:03.407566 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12681,"thread_start_us":12548,"threads_started":1}
W20260811 00:26:03.408861 4779 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50240
W20260811 00:26:03.409678 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 4617: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:26:03.409983 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 4617
I20260811 00:26:03.441898 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 4788
Aug 11 00:26:03 dist-test-slave-fd7l krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407961, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.120.2@KRBTEST.COM
I20260811 00:26:03.460485 5302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:03.442865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53888 (local address 127.2.120.2:33923)
0811 00:26:03.443539 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:03.443544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:03.443560 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:26:03.443616 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:03.443619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:03.443679 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:03.443808 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:03.445259 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:03.446054 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:03.447463 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:03.447921 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:03.448082 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:03.453097 (+ 5015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:03.453117 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:03.453138 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:03.453194 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:03.456694 (+ 3500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:03.457669 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:03.457685 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:03.457690 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:03.458001 (+ 311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:03.458667 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:03.458693 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:03.458697 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:03.459174 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:03.459460 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:03.459810 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 00:26:03.460026 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":391,"threads_started":1}
W20260811 00:26:03.461416 4941 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53888
W20260811 00:26:03.462203 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 4788: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:26:03.462402 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 4788
I20260811 00:26:03.492797 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 4952
Aug 11 00:26:03 dist-test-slave-fd7l krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407961, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.120.3@KRBTEST.COM
I20260811 00:26:03.511794 5303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:03.493834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39062 (local address 127.2.120.3:38583)
0811 00:26:03.494378 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:03.494383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:03.494399 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:26:03.494459 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:03.494463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:03.494540 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:03.494674 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:03.496162 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:03.497072 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:03.498425 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:03.498863 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:03.499021 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:03.504615 (+ 5594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:03.504634 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:03.504654 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:03.504710 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:03.507892 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:03.508983 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:03.508999 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:03.509007 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:03.509252 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:03.509806 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:03.509814 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:03.509820 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:03.510452 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:03.510771 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:03.511154 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 00:26:03.511451 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":259,"threads_started":1}
W20260811 00:26:03.512715 5101 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39062
W20260811 00:26:03.513582 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 4952: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:26:03.513808 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 4952
I20260811 00:26:03.548949 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 4500
I20260811 00:26:03.550422 4583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:03.672392 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 4500
2026-08-11T00:26:03Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9410 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:26:05 dist-test-slave-fd7l krb5kdc[5309](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:26:05 dist-test-slave-fd7l krb5kdc[5309](info): set up 2 sockets
Aug 11 00:26:05 dist-test-slave-fd7l krb5kdc[5309](info): commencing operation
krb5kdc: starting...
W20260811 00:26:06.622555 2528 mini_kdc.cc:121] Time spent starting KDC: real 2.896s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:26:06 dist-test-slave-fd7l krb5kdc[5309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:26:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:26:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:26:06.873502 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:41765
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:37199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:41765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:07.276371 5323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:07.276834 5323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:07.276938 5323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:07.285444 5323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:26:07.285602 5323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:07.285658 5323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:07.285714 5323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:26:07.285756 5323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:26:07.296993 5323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:41765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:41765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:07.298489 5323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:07.299896 5323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:07.314073 5328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:07.314458 5330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:07.314518 5332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:07.315387 5323 server_base.cc:1034] running on GCE node
I20260811 00:26:07.316447 5323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:07.318271 5323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:07.319530 5323 hybrid_clock.cc:648] HybridClock initialized: now 1786407967319466 us; error 55 us; skew 500 ppm
I20260811 00:26:07.320001 5323 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:07 dist-test-slave-fd7l krb5kdc[5309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407967, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:07.329895 5323 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:26:07.331912 5323 webserver.cc:460] Webserver started at http://127.2.120.62:41995/ using document root <none> and password file <none>
I20260811 00:26:07.332880 5323 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:07.333005 5323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:07.333370 5323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:07.337025 5323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "76c2d281a3c94dec908ef0779ce328b5"
format_stamp: "Formatted at 2026-08-11 00:26:07 on dist-test-slave-fd7l"
server_key: "b681e41ea09cacc6cc508b150bac1cb2"
server_key_iv: "a0848bf3c77561a365645dbf8056a58e"
server_key_version: "encryptionkey@0"
I20260811 00:26:07.338045 5323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "76c2d281a3c94dec908ef0779ce328b5"
format_stamp: "Formatted at 2026-08-11 00:26:07 on dist-test-slave-fd7l"
server_key: "b681e41ea09cacc6cc508b150bac1cb2"
server_key_iv: "a0848bf3c77561a365645dbf8056a58e"
server_key_version: "encryptionkey@0"
I20260811 00:26:07.344991 5323 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:26:07.349768 5340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:07.351509 5323 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:26:07.351805 5323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "76c2d281a3c94dec908ef0779ce328b5"
format_stamp: "Formatted at 2026-08-11 00:26:07 on dist-test-slave-fd7l"
server_key: "b681e41ea09cacc6cc508b150bac1cb2"
server_key_iv: "a0848bf3c77561a365645dbf8056a58e"
server_key_version: "encryptionkey@0"
I20260811 00:26:07.352065 5323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:07.378800 5323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:07.384008 5323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:07.384342 5323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:07.405985 5323 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:41765
I20260811 00:26:07.406021 5396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:41765 every 8 connection(s)
I20260811 00:26:07.407678 5323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:26:07.413236 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 5323
I20260811 00:26:07.413569 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:26:07.414494 2528 external_mini_cluster.cc:1428] Setting key 9cabce348ab686ece67aa13f21863698
I20260811 00:26:07.414501 5397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 00:26:07 dist-test-slave-fd7l krb5kdc[5309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:07.434386 5397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5: Bootstrap starting.
I20260811 00:26:07.439617 5397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:07.441421 5397 log.cc:824] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:07.445794 5397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5: No bootstrap required, opened a new log
I20260811 00:26:07.447088 5400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:07.417031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60516 (local address 127.2.120.62:41765)
0811 00:26:07.417987 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:07.418000 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:07.418047 (+ 47us) server_negotiation.cc:407] Connection header received
0811 00:26:07.421360 (+ 3313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:07.421398 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:07.422007 (+ 609us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:07.422498 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:07.425025 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:07.426690 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:07.428958 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:07.429874 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:07.430071 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:07.435876 (+ 5805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:07.435891 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:07.435915 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:07.435999 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:07.441220 (+ 5221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:07.442515 (+ 1295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:07.442525 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:07.442532 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:07.442749 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:07.443421 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:07.443430 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:07.443436 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:07.444220 (+ 784us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:07.444586 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:07.445096 (+ 510us) server_negotiation.cc:299] Negotiation successful
0811 00:26:07.445578 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":435,"threads_started":1}
I20260811 00:26:07.454864 5397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c2d281a3c94dec908ef0779ce328b5" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 41765 } }
I20260811 00:26:07.455312 5397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:07.455428 5397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76c2d281a3c94dec908ef0779ce328b5, State: Initialized, Role: FOLLOWER
I20260811 00:26:07.456032 5397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c2d281a3c94dec908ef0779ce328b5" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 41765 } }
I20260811 00:26:07.456373 5397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:26:07.456574 5397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:26:07.456790 5397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:07.459578 5397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c2d281a3c94dec908ef0779ce328b5" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 41765 } }
I20260811 00:26:07.460207 5397 leader_election.cc:302] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76c2d281a3c94dec908ef0779ce328b5; no voters:
I20260811 00:26:07.460871 5397 leader_election.cc:288] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:26:07.461051 5402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:07.462386 5402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5 [term 1 LEADER]: Becoming Leader. State: Replica: 76c2d281a3c94dec908ef0779ce328b5, State: Running, Role: LEADER
I20260811 00:26:07.463333 5402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c2d281a3c94dec908ef0779ce328b5" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 41765 } }
I20260811 00:26:07.463935 5397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:26:07.467691 5403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76c2d281a3c94dec908ef0779ce328b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c2d281a3c94dec908ef0779ce328b5" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 41765 } } }
I20260811 00:26:07.467125 5404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76c2d281a3c94dec908ef0779ce328b5. Latest consensus state: current_term: 1 leader_uuid: "76c2d281a3c94dec908ef0779ce328b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c2d281a3c94dec908ef0779ce328b5" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 41765 } } }
I20260811 00:26:07.469702 5403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5 [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:07.469916 5404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5 [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:07.474964 5410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:26:07.482945 5410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:26:07.494306 5410 catalog_manager.cc:1347] Generated new cluster ID: 84ca24a9778c435cac4b249269643b1c
I20260811 00:26:07.494452 5410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:26:07.548069 5410 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:26:07.549721 5410 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:26:07.558456 5410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76c2d281a3c94dec908ef0779ce328b5: Generated new TSK 0
I20260811 00:26:07.559612 5410 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
I20260811 00:26:07.639786 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:41765
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:37199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:08.012210 5426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:08.012614 5426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:08.012781 5426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:08.020262 5426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:08.020433 5426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:08.020648 5426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:26:08.031453 5426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:41765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:08.033089 5426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:08.034495 5426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:08.051400 5431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:08.051348 5432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:08.051317 5434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:08.052691 5426 server_base.cc:1034] running on GCE node
I20260811 00:26:08.053452 5426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:08.054735 5426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:08.056017 5426 hybrid_clock.cc:648] HybridClock initialized: now 1786407968055931 us; error 78 us; skew 500 ppm
I20260811 00:26:08.056478 5426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:08 dist-test-slave-fd7l krb5kdc[5309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407968, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:08.060690 5426 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:26:08.062673 5426 webserver.cc:460] Webserver started at http://127.2.120.1:33595/ using document root <none> and password file <none>
I20260811 00:26:08.063494 5426 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:08.063630 5426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:08.064073 5426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:08.067485 5426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "c0d31a005d54469995eed56f4dd4d0f2"
format_stamp: "Formatted at 2026-08-11 00:26:08 on dist-test-slave-fd7l"
server_key: "5f368f5c1f86eb55e650683e6fb7481a"
server_key_iv: "22f1259dd270a49aff0dffbdb8586703"
server_key_version: "encryptionkey@0"
I20260811 00:26:08.068425 5426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "c0d31a005d54469995eed56f4dd4d0f2"
format_stamp: "Formatted at 2026-08-11 00:26:08 on dist-test-slave-fd7l"
server_key: "5f368f5c1f86eb55e650683e6fb7481a"
server_key_iv: "22f1259dd270a49aff0dffbdb8586703"
server_key_version: "encryptionkey@0"
I20260811 00:26:08.074529 5426 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 00:26:08.079282 5445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:08.081914 5426 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 00:26:08.082262 5426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "c0d31a005d54469995eed56f4dd4d0f2"
format_stamp: "Formatted at 2026-08-11 00:26:08 on dist-test-slave-fd7l"
server_key: "5f368f5c1f86eb55e650683e6fb7481a"
server_key_iv: "22f1259dd270a49aff0dffbdb8586703"
server_key_version: "encryptionkey@0"
I20260811 00:26:08.082579 5426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:08.106643 5426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:08.111931 5426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:08.112277 5426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:08.113478 5426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:08.115106 5426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:08.115226 5426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:08.115384 5426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:08.115460 5426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:08.153090 5426 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:43071
I20260811 00:26:08.153189 5569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:43071 every 8 connection(s)
I20260811 00:26:08.154505 5426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:26:08.158000 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 5426
I20260811 00:26:08.158250 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:26:08.159209 2528 external_mini_cluster.cc:1428] Setting key 751ca57635acc17fcc7a4214459d6230
Aug 11 00:26:08 dist-test-slave-fd7l krb5kdc[5309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407968, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:08.178005 5572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:08.157675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:47953 (local address 127.2.120.62:41765)
0811 00:26:08.158227 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:08.158233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:08.159663 (+ 1430us) server_negotiation.cc:407] Connection header received
0811 00:26:08.162311 (+ 2648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:08.162317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:08.162410 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:08.162556 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:08.164217 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:08.165259 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:08.167055 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:08.167513 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:08.167670 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:08.171826 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:08.171832 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:08.171835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:08.171872 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:08.174561 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:08.175465 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:08.175472 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:08.175476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:08.175621 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:08.176098 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:08.176105 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:08.176110 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:08.176396 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:08.176604 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:08.177380 (+ 776us) server_negotiation.cc:299] Negotiation successful
0811 00:26:08.177636 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":272,"threads_started":1}
I20260811 00:26:08.179236 5573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:08.158509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:41765 (local address 127.2.120.1:47953)
0811 00:26:08.159493 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:08.159534 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:08.162002 (+ 2468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:08.162823 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:08.162833 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:08.163407 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:08.164028 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:08.164046 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:08.165459 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:08.165464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:08.166870 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:08.166895 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:08.167779 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:08.167787 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:08.168067 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:08.168941 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:08.168969 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:08.171528 (+ 2559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:08.174777 (+ 3249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:08.174789 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:08.174811 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:08.175314 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:08.175748 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:08.175753 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:08.175758 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:08.175972 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:08.176567 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:08.176576 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:08.176972 (+ 396us) client_negotiation.cc:769] Sending connection context
0811 00:26:08.177282 (+ 310us) client_negotiation.cc:240] Negotiation successful
0811 00:26:08.177863 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":400,"threads_started":1}
WARNING: no policy specified for kudu/127.2.120.2@KRBTEST.COM; defaulting to no policy
I20260811 00:26:08.181319 5570 heartbeater.cc:347] Connected to a master server at 127.2.120.62:41765
I20260811 00:26:08.181696 5570 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:08.182787 5570 heartbeater.cc:511] Master 127.2.120.62:41765 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.2@KRBTEST.COM" created.
I20260811 00:26:08.185613 5358 ts_manager.cc:194] Registered new tserver with Master: c0d31a005d54469995eed56f4dd4d0f2 (127.2.120.1:43071)
I20260811 00:26:08.188037 5358 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:47953
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
I20260811 00:26:08.217505 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:41765
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:37199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:08.573226 5578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:08.573590 5578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:08.573761 5578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:08.581390 5578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:08.581570 5578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:08.581768 5578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:26:08.592530 5578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:41765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:08.593968 5578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:08.595402 5578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:08.610455 5584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:08.610352 5587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:08.610432 5585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:08.621052 5578 server_base.cc:1034] running on GCE node
I20260811 00:26:08.621922 5578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:08.623303 5578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:08.624651 5578 hybrid_clock.cc:648] HybridClock initialized: now 1786407968624669 us; error 111 us; skew 500 ppm
I20260811 00:26:08.625070 5578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:08 dist-test-slave-fd7l krb5kdc[5309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407968, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:08.628894 5578 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:26:08.630769 5578 webserver.cc:460] Webserver started at http://127.2.120.2:46719/ using document root <none> and password file <none>
I20260811 00:26:08.631560 5578 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:08.631675 5578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:08.631961 5578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:08.635371 5578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "edabc628575545658421a0b157fdc855"
format_stamp: "Formatted at 2026-08-11 00:26:08 on dist-test-slave-fd7l"
server_key: "e8da465504cb3bd92e21078ab11ca44c"
server_key_iv: "9fdef9e1f6cb48b68c1100a1512dbeeb"
server_key_version: "encryptionkey@0"
I20260811 00:26:08.636277 5578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "edabc628575545658421a0b157fdc855"
format_stamp: "Formatted at 2026-08-11 00:26:08 on dist-test-slave-fd7l"
server_key: "e8da465504cb3bd92e21078ab11ca44c"
server_key_iv: "9fdef9e1f6cb48b68c1100a1512dbeeb"
server_key_version: "encryptionkey@0"
I20260811 00:26:08.642406 5578 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260811 00:26:08.646467 5595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:08.648018 5578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:26:08.648172 5578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "edabc628575545658421a0b157fdc855"
format_stamp: "Formatted at 2026-08-11 00:26:08 on dist-test-slave-fd7l"
server_key: "e8da465504cb3bd92e21078ab11ca44c"
server_key_iv: "9fdef9e1f6cb48b68c1100a1512dbeeb"
server_key_version: "encryptionkey@0"
I20260811 00:26:08.648362 5578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:08.662364 5578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:08.667585 5578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:08.667878 5578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:08.669353 5578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:08.670845 5578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:08.670928 5578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:08.671015 5578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:08.671062 5578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:08.707238 5578 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:36503
I20260811 00:26:08.707278 5727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:36503 every 8 connection(s)
I20260811 00:26:08.708630 5578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:26:08.712716 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 5578
I20260811 00:26:08.712927 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:26:08.713475 2528 external_mini_cluster.cc:1428] Setting key c2f06c7f2ee111f3040b2da09b368e66
Aug 11 00:26:08 dist-test-slave-fd7l krb5kdc[5309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407968, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:08.734253 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:08.711626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:33003 (local address 127.2.120.62:41765)
0811 00:26:08.712128 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:08.712132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:08.713218 (+ 1086us) server_negotiation.cc:407] Connection header received
0811 00:26:08.715382 (+ 2164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:08.715389 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:08.715484 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:08.715606 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:08.717896 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:08.718646 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:08.721518 (+ 2872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:08.722025 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:08.722150 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:08.726209 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:08.726218 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:08.726227 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:08.726282 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:08.730174 (+ 3892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:08.731182 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:08.731191 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:08.731197 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:08.731363 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:08.731878 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:08.731885 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:08.731890 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:08.732344 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:08.732542 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:08.733431 (+ 889us) server_negotiation.cc:299] Negotiation successful
0811 00:26:08.733779 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":291,"threads_started":1}
I20260811 00:26:08.734961 5732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:08.712073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:41765 (local address 127.2.120.2:33003)
0811 00:26:08.713019 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:08.713081 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:08.715041 (+ 1960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:08.715892 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:08.715904 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:08.716790 (+ 886us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:08.717691 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:08.717721 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:08.718815 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:08.718824 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:08.721302 (+ 2478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:08.721322 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:08.722303 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:08.722313 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:08.722639 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:08.723406 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:08.723432 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:08.725866 (+ 2434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:08.730405 (+ 4539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:08.730416 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:08.730432 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:08.731008 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:08.731484 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:08.731507 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:08.731509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:08.731728 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:08.732579 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:08.732592 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:08.732979 (+ 387us) client_negotiation.cc:769] Sending connection context
0811 00:26:08.733245 (+ 266us) client_negotiation.cc:240] Negotiation successful
0811 00:26:08.733746 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":418,"threads_started":1}
WARNING: no policy specified for kudu/127.2.120.3@KRBTEST.COM; defaulting to no policy
I20260811 00:26:08.736676 5728 heartbeater.cc:347] Connected to a master server at 127.2.120.62:41765
I20260811 00:26:08.737013 5728 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:08.737767 5728 heartbeater.cc:511] Master 127.2.120.62:41765 requested a full tablet report, sending...
I20260811 00:26:08.739466 5358 ts_manager.cc:194] Registered new tserver with Master: edabc628575545658421a0b157fdc855 (127.2.120.2:36503)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.3@KRBTEST.COM" created.
I20260811 00:26:08.740665 5358 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:33003
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
I20260811 00:26:08.767853 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:41765
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:37199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:09.130237 5735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:09.130522 5735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:09.130616 5735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:09.138062 5735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:09.138192 5735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:09.138307 5735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:26:09.148663 5735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:41765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:09.150112 5735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:09.151401 5735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:09.167097 5740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:09.167124 5741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:09.167332 5744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:09.169245 5735 server_base.cc:1034] running on GCE node
I20260811 00:26:09.169960 5735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:09.171036 5735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:09.172312 5735 hybrid_clock.cc:648] HybridClock initialized: now 1786407969172248 us; error 57 us; skew 500 ppm
I20260811 00:26:09.172784 5735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:09 dist-test-slave-fd7l krb5kdc[5309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407969, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:09.176352 5735 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:26:09.178153 5735 webserver.cc:460] Webserver started at http://127.2.120.3:39977/ using document root <none> and password file <none>
I20260811 00:26:09.179062 5735 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:09.179224 5735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:09.179569 5735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:09.182781 5735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "3c1b7d8514804f5e991ac517717c5cf4"
format_stamp: "Formatted at 2026-08-11 00:26:09 on dist-test-slave-fd7l"
server_key: "f7b94c4fb08c302cc8545d15d607590d"
server_key_iv: "31be499f8d0f5b113d781a0d55219630"
server_key_version: "encryptionkey@0"
I20260811 00:26:09.183609 5735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "3c1b7d8514804f5e991ac517717c5cf4"
format_stamp: "Formatted at 2026-08-11 00:26:09 on dist-test-slave-fd7l"
server_key: "f7b94c4fb08c302cc8545d15d607590d"
server_key_iv: "31be499f8d0f5b113d781a0d55219630"
server_key_version: "encryptionkey@0"
I20260811 00:26:09.189275 5735 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 00:26:09.192703 5570 heartbeater.cc:503] Master 127.2.120.62:41765 was elected leader, sending a full tablet report...
I20260811 00:26:09.192986 5751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:09.194572 5735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 00:26:09.194702 5735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "3c1b7d8514804f5e991ac517717c5cf4"
format_stamp: "Formatted at 2026-08-11 00:26:09 on dist-test-slave-fd7l"
server_key: "f7b94c4fb08c302cc8545d15d607590d"
server_key_iv: "31be499f8d0f5b113d781a0d55219630"
server_key_version: "encryptionkey@0"
I20260811 00:26:09.194868 5735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:09.222450 5735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:09.228024 5735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:09.228320 5735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:09.229386 5735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:09.230872 5735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:09.230948 5735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:09.231127 5735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:09.231201 5735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:09.268821 5735 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:37473
I20260811 00:26:09.268872 5884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:37473 every 8 connection(s)
I20260811 00:26:09.270210 5735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:26:09.272270 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 5735
I20260811 00:26:09.272497 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:26:09.273104 2528 external_mini_cluster.cc:1428] Setting key dd9366659aa61a06e27e773ffc2d7327
Aug 11 00:26:09 dist-test-slave-fd7l krb5kdc[5309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407969, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:09.294255 5887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:09.273252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:39329 (local address 127.2.120.62:41765)
0811 00:26:09.273764 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:09.273772 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:09.275034 (+ 1262us) server_negotiation.cc:407] Connection header received
0811 00:26:09.277350 (+ 2316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:09.277354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:09.277427 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:09.277524 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:09.279320 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:09.280113 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:09.281942 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:09.282398 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:09.282520 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:09.286694 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:09.286700 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:09.286704 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:09.286746 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:09.290556 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:09.291554 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:09.291559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:09.291562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:09.291711 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:09.292237 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:09.292242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:09.292245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:09.292767 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:09.292950 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:09.293743 (+ 793us) server_negotiation.cc:299] Negotiation successful
0811 00:26:09.293966 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":277,"threads_started":1}
I20260811 00:26:09.295292 5888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:09.273904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:41765 (local address 127.2.120.3:39329)
0811 00:26:09.274827 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:09.274891 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:09.277067 (+ 2176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:09.277761 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:09.277770 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:09.278441 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:09.279143 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:09.279166 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:09.280284 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:09.280289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:09.281745 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:09.281766 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:09.282683 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:09.282689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:09.282944 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:09.283763 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:09.283790 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:09.286304 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:09.290751 (+ 4447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:09.290761 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:09.290779 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:09.291376 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:09.291823 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:09.291827 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:09.291829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:09.292049 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:09.292902 (+ 853us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:09.292909 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:09.293310 (+ 401us) client_negotiation.cc:769] Sending connection context
0811 00:26:09.293574 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 00:26:09.294054 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":367,"threads_started":1}
I20260811 00:26:09.297293 5885 heartbeater.cc:347] Connected to a master server at 127.2.120.62:41765
I20260811 00:26:09.297690 5885 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:09.298784 5885 heartbeater.cc:511] Master 127.2.120.62:41765 requested a full tablet report, sending...
I20260811 00:26:09.300943 5358 ts_manager.cc:194] Registered new tserver with Master: 3c1b7d8514804f5e991ac517717c5cf4 (127.2.120.3:37473)
I20260811 00:26:09.302040 5358 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:39329
I20260811 00:26:09.302966 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 00:26:09.341640 5887 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:26:09.338819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60530 (local address 127.2.120.62:41765)
0811 00:26:09.339109 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:09.339119 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:09.339452 (+ 333us) server_negotiation.cc:407] Connection header received
0811 00:26:09.339692 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:09.339702 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:09.339786 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:09.339908 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:09.341339 (+ 1431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60530: BlockingRecv error: recv got EOF from 127.0.0.1:60530 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
I20260811 00:26:09.345656 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 5426
W20260811 00:26:09.349182 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 5426: Not authorized: Client connection negotiation failed: client connection to 127.2.120.1:43071: server requires authentication, but client does not have Kerberos credentials available
I20260811 00:26:09.349432 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 5426
I20260811 00:26:09.370795 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 5578
W20260811 00:26:09.373679 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 5578: Not authorized: Client connection negotiation failed: client connection to 127.2.120.2:36503: server requires authentication, but client does not have Kerberos credentials available
I20260811 00:26:09.373863 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 5578
I20260811 00:26:09.394469 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 5735
W20260811 00:26:09.397794 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 5735: Not authorized: Client connection negotiation failed: client connection to 127.2.120.3:37473: server requires authentication, but client does not have Kerberos credentials available
I20260811 00:26:09.397921 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 5735
I20260811 00:26:09.417917 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 5323
I20260811 00:26:09.419284 5392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:09.547189 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 5323
2026-08-11T00:26:09Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (5868 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:26:09 dist-test-slave-fd7l krb5kdc[5904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:26:09 dist-test-slave-fd7l krb5kdc[5904](info): set up 2 sockets
Aug 11 00:26:09 dist-test-slave-fd7l krb5kdc[5904](info): commencing operation
krb5kdc: starting...
W20260811 00:26:10.772048 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:26:10 dist-test-slave-fd7l krb5kdc[5904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:26:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:26:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:26:11.041319 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:46363
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:33513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:46363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:11.419618 5920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:11.419932 5920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:11.420090 5920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:11.427553 5920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:26:11.427673 5920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:11.427718 5920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:11.427771 5920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:26:11.427870 5920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:26:11.438262 5920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:33513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:46363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:46363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:11.439687 5920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:11.441143 5920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:11.455278 5929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:11.455327 5926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:11.455238 5927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:11.456216 5920 server_base.cc:1034] running on GCE node
I20260811 00:26:11.457185 5920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:11.458894 5920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:11.460198 5920 hybrid_clock.cc:648] HybridClock initialized: now 1786407971460147 us; error 51 us; skew 500 ppm
I20260811 00:26:11.460780 5920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:11 dist-test-slave-fd7l krb5kdc[5904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407971, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:11.465909 5920 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:26:11.469110 5920 webserver.cc:460] Webserver started at http://127.2.120.62:35349/ using document root <none> and password file <none>
I20260811 00:26:11.470206 5920 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:11.470369 5920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:11.470818 5920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:11.475098 5920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "ebd22b38f637452fabb232af9bff8015"
format_stamp: "Formatted at 2026-08-11 00:26:11 on dist-test-slave-fd7l"
server_key: "cea9c4edce03f627c9ca0e56224740ec"
server_key_iv: "9a24c63a3bd0387866c430b7dedc02b5"
server_key_version: "encryptionkey@0"
I20260811 00:26:11.476087 5920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "ebd22b38f637452fabb232af9bff8015"
format_stamp: "Formatted at 2026-08-11 00:26:11 on dist-test-slave-fd7l"
server_key: "cea9c4edce03f627c9ca0e56224740ec"
server_key_iv: "9a24c63a3bd0387866c430b7dedc02b5"
server_key_version: "encryptionkey@0"
I20260811 00:26:11.482024 5920 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 00:26:11.486045 5938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:11.487864 5920 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260811 00:26:11.488142 5920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "ebd22b38f637452fabb232af9bff8015"
format_stamp: "Formatted at 2026-08-11 00:26:11 on dist-test-slave-fd7l"
server_key: "cea9c4edce03f627c9ca0e56224740ec"
server_key_iv: "9a24c63a3bd0387866c430b7dedc02b5"
server_key_version: "encryptionkey@0"
I20260811 00:26:11.488408 5920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:11.503427 5920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:11.508419 5920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:11.508810 5920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:11.530897 5920 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:46363
I20260811 00:26:11.530946 5993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:46363 every 8 connection(s)
I20260811 00:26:11.532593 5920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:26:11.537117 5994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:11.538156 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 5920
I20260811 00:26:11.538386 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:26:11.539258 2528 external_mini_cluster.cc:1428] Setting key e483eec7e429dc0de3e0247c086d6ac6
I20260811 00:26:11.551913 5994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015: Bootstrap starting.
Aug 11 00:26:11 dist-test-slave-fd7l krb5kdc[5904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:11.555866 5994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:11.557251 5994 log.cc:824] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:11.560693 5994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015: No bootstrap required, opened a new log
I20260811 00:26:11.565789 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:11.541572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56682 (local address 127.2.120.62:46363)
0811 00:26:11.542491 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:11.542504 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:11.542556 (+ 52us) server_negotiation.cc:407] Connection header received
0811 00:26:11.544427 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:11.544459 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:11.545078 (+ 619us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:11.545563 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:11.547279 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:11.548789 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:11.550216 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:11.551005 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:11.551262 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:11.556311 (+ 5049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:11.556324 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:11.556343 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:11.556398 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:11.560117 (+ 3719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:11.561234 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:11.561242 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:11.561254 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:11.561460 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:11.561953 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:11.561957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:11.561959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:11.562626 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:11.562890 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:11.563984 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0811 00:26:11.564473 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":439,"threads_started":1}
I20260811 00:26:11.570856 5994 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebd22b38f637452fabb232af9bff8015" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 46363 } }
I20260811 00:26:11.571393 5994 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:11.571555 5994 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebd22b38f637452fabb232af9bff8015, State: Initialized, Role: FOLLOWER
I20260811 00:26:11.571686 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:26:11.572315 5994 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebd22b38f637452fabb232af9bff8015" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 46363 } }
I20260811 00:26:11.572640 5994 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:26:11.572800 5994 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:26:11.573000 5994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:11.575444 5994 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebd22b38f637452fabb232af9bff8015" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 46363 } }
I20260811 00:26:11.576107 5994 leader_election.cc:302] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebd22b38f637452fabb232af9bff8015; no voters:
I20260811 00:26:11.576874 5994 leader_election.cc:288] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:26:11.577070 5999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:11.578347 5999 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015 [term 1 LEADER]: Becoming Leader. State: Replica: ebd22b38f637452fabb232af9bff8015, State: Running, Role: LEADER
I20260811 00:26:11.579092 5999 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebd22b38f637452fabb232af9bff8015" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 46363 } }
I20260811 00:26:11.579550 5994 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:26:11.582803 6000 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebd22b38f637452fabb232af9bff8015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebd22b38f637452fabb232af9bff8015" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 46363 } } }
I20260811 00:26:11.582755 6001 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebd22b38f637452fabb232af9bff8015. Latest consensus state: current_term: 1 leader_uuid: "ebd22b38f637452fabb232af9bff8015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebd22b38f637452fabb232af9bff8015" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 46363 } } }
I20260811 00:26:11.583379 6000 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015 [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:11.583379 6001 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015 [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:11.587397 6006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:26:11.595191 6006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:26:11.609803 6006 catalog_manager.cc:1347] Generated new cluster ID: a2fa893c630f46e59d634580e77c0e36
I20260811 00:26:11.609970 6006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:26:11.634014 6006 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:26:11.636034 6006 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:26:11.645120 6006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebd22b38f637452fabb232af9bff8015: Generated new TSK 0
I20260811 00:26:11.645804 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:11.635821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56698 (local address 127.2.120.62:46363)
0811 00:26:11.636037 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:11.636043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:11.636762 (+ 719us) server_negotiation.cc:407] Connection header received
0811 00:26:11.636976 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:11.636984 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:11.637100 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:11.637256 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:11.638657 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:11.639995 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:11.641751 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:11.642287 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:11.642469 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:11.643958 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:11.643967 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:11.643970 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:11.644814 (+ 844us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:11.644911 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:11.644954 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:11.645126 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:11.645273 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 00:26:11.645395 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 00:26:11.646391 6006 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:26:11.658330 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 5920
I20260811 00:26:11.659906 5988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:11.753501 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 5920
2026-08-11T00:26:11Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2200 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:26:11 dist-test-slave-fd7l krb5kdc[6030](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:26:11 dist-test-slave-fd7l krb5kdc[6030](info): set up 2 sockets
Aug 11 00:26:11 dist-test-slave-fd7l krb5kdc[6030](info): commencing operation
krb5kdc: starting...
W20260811 00:26:12.983414 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:26:13 dist-test-slave-fd7l krb5kdc[6030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:26:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:26:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:26:13.257189 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:41027
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:38691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:41027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:13.622393 6044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:13.622723 6044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:13.622815 6044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:13.630129 6044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:26:13.630252 6044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:13.630299 6044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:13.630446 6044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:26:13.630537 6044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:26:13.640980 6044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:38691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:41027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:41027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:13.642535 6044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:13.643976 6044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:13.656993 6050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:13.657044 6049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:13.657075 6053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:13.657811 6044 server_base.cc:1034] running on GCE node
I20260811 00:26:13.658847 6044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:13.660435 6044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:13.661736 6044 hybrid_clock.cc:648] HybridClock initialized: now 1786407973661681 us; error 47 us; skew 500 ppm
I20260811 00:26:13.662191 6044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:13 dist-test-slave-fd7l krb5kdc[6030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407973, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:13.671309 6044 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:26:13.673465 6044 webserver.cc:460] Webserver started at http://127.2.120.62:33131/ using document root <none> and password file <none>
I20260811 00:26:13.674297 6044 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:13.674471 6044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:13.674822 6044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:13.678323 6044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "989c5660932849b186ca82299c598354"
format_stamp: "Formatted at 2026-08-11 00:26:13 on dist-test-slave-fd7l"
server_key: "07643cbf2e6fa0739a32afaf80fce732"
server_key_iv: "5b081eecf7383e29a53c13843b9efb51"
server_key_version: "encryptionkey@0"
I20260811 00:26:13.679257 6044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "989c5660932849b186ca82299c598354"
format_stamp: "Formatted at 2026-08-11 00:26:13 on dist-test-slave-fd7l"
server_key: "07643cbf2e6fa0739a32afaf80fce732"
server_key_iv: "5b081eecf7383e29a53c13843b9efb51"
server_key_version: "encryptionkey@0"
I20260811 00:26:13.685670 6044 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260811 00:26:13.690004 6061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:13.691697 6044 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:26:13.691962 6044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "989c5660932849b186ca82299c598354"
format_stamp: "Formatted at 2026-08-11 00:26:13 on dist-test-slave-fd7l"
server_key: "07643cbf2e6fa0739a32afaf80fce732"
server_key_iv: "5b081eecf7383e29a53c13843b9efb51"
server_key_version: "encryptionkey@0"
I20260811 00:26:13.692219 6044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:13.710389 6044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:13.715520 6044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:13.715874 6044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:13.739519 6044 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:41027
I20260811 00:26:13.739563 6119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:41027 every 8 connection(s)
I20260811 00:26:13.741361 6044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:26:13.743431 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 6044
I20260811 00:26:13.743750 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:26:13.744788 2528 external_mini_cluster.cc:1428] Setting key 2d4e169504458a59b0188585aad6cd18
I20260811 00:26:13.747992 6120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 00:26:13 dist-test-slave-fd7l krb5kdc[6030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:13.766266 6120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354: Bootstrap starting.
I20260811 00:26:13.770255 6120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:13.771492 6120 log.cc:824] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:13.775257 6120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354: No bootstrap required, opened a new log
I20260811 00:26:13.775585 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:13.747763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41012 (local address 127.2.120.62:41027)
0811 00:26:13.749320 (+ 1557us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:13.749334 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:13.749379 (+ 45us) server_negotiation.cc:407] Connection header received
0811 00:26:13.751091 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:13.751119 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:13.751602 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:13.752003 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:13.754305 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:13.756398 (+ 2093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:13.759091 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:13.759987 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:13.760211 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:13.766611 (+ 6400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:13.766638 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:13.766671 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:13.766737 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:13.770310 (+ 3573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:13.771330 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:13.771338 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:13.771345 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:13.771567 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:13.772048 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:13.772053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:13.772057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:13.773051 (+ 994us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:13.773417 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:13.773928 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 00:26:13.774433 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1285,"thread_start_us":958,"threads_started":1}
I20260811 00:26:13.785030 6120 raft_consensus.cc:348] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989c5660932849b186ca82299c598354" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 41027 } }
I20260811 00:26:13.785561 6120 raft_consensus.cc:374] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:13.785691 6120 raft_consensus.cc:729] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 989c5660932849b186ca82299c598354, State: Initialized, Role: FOLLOWER
I20260811 00:26:13.786463 6120 consensus_queue.cc:260] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989c5660932849b186ca82299c598354" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 41027 } }
I20260811 00:26:13.786784 6120 raft_consensus.cc:388] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:26:13.786959 6120 raft_consensus.cc:482] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:26:13.787184 6120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:13.790990 6120 raft_consensus.cc:504] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989c5660932849b186ca82299c598354" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 41027 } }
I20260811 00:26:13.792220 6120 leader_election.cc:302] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 989c5660932849b186ca82299c598354; no voters:
I20260811 00:26:13.793553 6120 leader_election.cc:288] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:26:13.793953 6125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:13.796461 6125 raft_consensus.cc:686] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354 [term 1 LEADER]: Becoming Leader. State: Replica: 989c5660932849b186ca82299c598354, State: Running, Role: LEADER
I20260811 00:26:13.797725 6125 consensus_queue.cc:237] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989c5660932849b186ca82299c598354" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 41027 } }
I20260811 00:26:13.798218 6120 sys_catalog.cc:564] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:26:13.803820 6127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 989c5660932849b186ca82299c598354. Latest consensus state: current_term: 1 leader_uuid: "989c5660932849b186ca82299c598354" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989c5660932849b186ca82299c598354" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 41027 } } }
I20260811 00:26:13.804539 6127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354 [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:13.804301 6126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "989c5660932849b186ca82299c598354" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989c5660932849b186ca82299c598354" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 41027 } } }
I20260811 00:26:13.804834 6126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354 [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:13.809819 6135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:26:13.819664 6135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:26:13.832477 6135 catalog_manager.cc:1347] Generated new cluster ID: b73cc91ed4264158b0c4580ee060a218
I20260811 00:26:13.832674 6135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:26:13.870283 6135 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:26:13.872737 6135 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:26:13.884714 6135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 989c5660932849b186ca82299c598354: Generated new TSK 0
I20260811 00:26:13.885998 6135 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
I20260811 00:26:13.958776 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:41027
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:38691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:14.359165 6149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:14.359743 6149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:14.359933 6149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:14.367942 6149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:14.368093 6149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:14.368343 6149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:26:14.379092 6149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:38691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:41027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:14.380656 6149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:14.382155 6149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:14.397496 6157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:14.397526 6159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:14.397780 6156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:14.398591 6149 server_base.cc:1034] running on GCE node
I20260811 00:26:14.399341 6149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:14.400591 6149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:14.401865 6149 hybrid_clock.cc:648] HybridClock initialized: now 1786407974401813 us; error 37 us; skew 500 ppm
I20260811 00:26:14.402333 6149 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:14 dist-test-slave-fd7l krb5kdc[6030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407974, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:14.407837 6149 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:26:14.410312 6149 webserver.cc:460] Webserver started at http://127.2.120.1:40465/ using document root <none> and password file <none>
I20260811 00:26:14.411240 6149 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:14.411429 6149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:14.411815 6149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:14.415447 6149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "db5a25aa4aa54752bc411bce22aad009"
format_stamp: "Formatted at 2026-08-11 00:26:14 on dist-test-slave-fd7l"
server_key: "4362fac012df56cdeb7456d43f95bb7a"
server_key_iv: "cb1b10b61a723a82dae4504e6039e20d"
server_key_version: "encryptionkey@0"
I20260811 00:26:14.416637 6149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "db5a25aa4aa54752bc411bce22aad009"
format_stamp: "Formatted at 2026-08-11 00:26:14 on dist-test-slave-fd7l"
server_key: "4362fac012df56cdeb7456d43f95bb7a"
server_key_iv: "cb1b10b61a723a82dae4504e6039e20d"
server_key_version: "encryptionkey@0"
I20260811 00:26:14.424022 6149 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.002s
I20260811 00:26:14.429081 6168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:14.430938 6149 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:26:14.431246 6149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "db5a25aa4aa54752bc411bce22aad009"
format_stamp: "Formatted at 2026-08-11 00:26:14 on dist-test-slave-fd7l"
server_key: "4362fac012df56cdeb7456d43f95bb7a"
server_key_iv: "cb1b10b61a723a82dae4504e6039e20d"
server_key_version: "encryptionkey@0"
I20260811 00:26:14.431550 6149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:14.454252 6149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:14.461647 6149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:14.462121 6149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:14.463821 6149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:14.465879 6149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:14.466015 6149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:14.466209 6149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:14.466300 6149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:14.513931 6149 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:46361
I20260811 00:26:14.513985 6298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:46361 every 8 connection(s)
I20260811 00:26:14.515614 6149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:26:14.519405 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 6149
I20260811 00:26:14.519629 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:26:14.520601 2528 external_mini_cluster.cc:1428] Setting key 6948d0ea38f57ce7c15e7cfe15bf9150
Aug 11 00:26:14 dist-test-slave-fd7l krb5kdc[6030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407974, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:14.546190 6302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:14.519167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:39859 (local address 127.2.120.62:41027)
0811 00:26:14.519738 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:14.519745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:14.521686 (+ 1941us) server_negotiation.cc:407] Connection header received
0811 00:26:14.524865 (+ 3179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:14.524875 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:14.524993 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:14.525148 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:14.527927 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:14.529413 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:14.531673 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:14.532152 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:14.532331 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:14.538807 (+ 6476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:14.538813 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:14.538817 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:14.538856 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:14.541901 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:14.543226 (+ 1325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:14.543231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:14.543234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:14.543376 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:14.544024 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:14.544030 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:14.544032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:14.544359 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:14.544497 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:14.545659 (+ 1162us) server_negotiation.cc:299] Negotiation successful
0811 00:26:14.545892 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":297,"threads_started":1}
WARNING: no policy specified for kudu/127.2.120.2@KRBTEST.COM; defaulting to no policy
I20260811 00:26:14.547643 6303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:14.519776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:41027 (local address 127.2.120.1:39859)
0811 00:26:14.521275 (+ 1499us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:14.521372 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:14.524251 (+ 2879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:14.525513 (+ 1262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:14.525525 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:14.526654 (+ 1129us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:14.527603 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:14.527636 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:14.529746 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:14.529752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:14.531406 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:14.531429 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:14.532496 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:14.532532 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:14.532960 (+ 428us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:14.533940 (+ 980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:14.533984 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:14.538447 (+ 4463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:14.542148 (+ 3701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:14.542162 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:14.542188 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:14.543040 (+ 852us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:14.543519 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:14.543524 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:14.543528 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:14.543826 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:14.544547 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:14.544559 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:14.545121 (+ 562us) client_negotiation.cc:769] Sending connection context
0811 00:26:14.545512 (+ 391us) client_negotiation.cc:240] Negotiation successful
0811 00:26:14.546185 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"thread_start_us":437,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.2@KRBTEST.COM" created.
I20260811 00:26:14.550161 6299 heartbeater.cc:347] Connected to a master server at 127.2.120.62:41027
I20260811 00:26:14.550663 6299 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:14.551930 6299 heartbeater.cc:511] Master 127.2.120.62:41027 requested a full tablet report, sending...
I20260811 00:26:14.554864 6080 ts_manager.cc:194] Registered new tserver with Master: db5a25aa4aa54752bc411bce22aad009 (127.2.120.1:46361)
I20260811 00:26:14.556469 6080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:39859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
I20260811 00:26:14.587882 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:41027
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:38691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:15.004478 6306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:15.004952 6306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:15.005143 6306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:15.014931 6306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:15.015189 6306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:15.015421 6306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:26:15.027442 6306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:38691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:41027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:15.029080 6306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:15.030654 6306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:15.047434 6311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:15.047287 6312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:15.047439 6314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:15.049054 6306 server_base.cc:1034] running on GCE node
I20260811 00:26:15.049990 6306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:15.051654 6306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:15.052942 6306 hybrid_clock.cc:648] HybridClock initialized: now 1786407975052899 us; error 43 us; skew 500 ppm
I20260811 00:26:15.053367 6306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:15 dist-test-slave-fd7l krb5kdc[6030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407975, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:15.057093 6306 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:26:15.058883 6306 webserver.cc:460] Webserver started at http://127.2.120.2:33839/ using document root <none> and password file <none>
I20260811 00:26:15.059693 6306 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:15.059854 6306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:15.060199 6306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:15.063453 6306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "027c15e08fee4eb5bf295aed05618116"
format_stamp: "Formatted at 2026-08-11 00:26:15 on dist-test-slave-fd7l"
server_key: "564bb3078bf93b604998c75c92c91083"
server_key_iv: "18a7f9459fedcd19e6c856a9c04c5608"
server_key_version: "encryptionkey@0"
I20260811 00:26:15.064327 6306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "027c15e08fee4eb5bf295aed05618116"
format_stamp: "Formatted at 2026-08-11 00:26:15 on dist-test-slave-fd7l"
server_key: "564bb3078bf93b604998c75c92c91083"
server_key_iv: "18a7f9459fedcd19e6c856a9c04c5608"
server_key_version: "encryptionkey@0"
I20260811 00:26:15.072321 6306 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260811 00:26:15.077571 6321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:15.079676 6306 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:26:15.079896 6306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "027c15e08fee4eb5bf295aed05618116"
format_stamp: "Formatted at 2026-08-11 00:26:15 on dist-test-slave-fd7l"
server_key: "564bb3078bf93b604998c75c92c91083"
server_key_iv: "18a7f9459fedcd19e6c856a9c04c5608"
server_key_version: "encryptionkey@0"
I20260811 00:26:15.080142 6306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:15.113826 6306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:15.119701 6306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:15.120160 6306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:15.121517 6306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:15.123293 6306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:15.123468 6306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:15.123647 6306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:15.123737 6306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:15.179546 6306 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:34293
I20260811 00:26:15.179625 6450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:34293 every 8 connection(s)
I20260811 00:26:15.182195 6306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:26:15.189663 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 6306
I20260811 00:26:15.190049 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:26:15.191027 2528 external_mini_cluster.cc:1428] Setting key 7c61992da1d3114a63b2ed76b8e33aa9
Aug 11 00:26:15 dist-test-slave-fd7l krb5kdc[6030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407975, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:15.221345 6453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:15.191880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:51517 (local address 127.2.120.62:41027)
0811 00:26:15.193549 (+ 1669us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:15.193562 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:15.196653 (+ 3091us) server_negotiation.cc:407] Connection header received
0811 00:26:15.200757 (+ 4104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:15.200767 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:15.200898 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:15.201061 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:15.203331 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:15.204091 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:15.206131 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:15.206685 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:15.206833 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:15.212155 (+ 5322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:15.212166 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:15.212174 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:15.212234 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:15.216611 (+ 4377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:15.217763 (+ 1152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:15.217772 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:15.217779 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:15.217979 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:15.218533 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:15.218540 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:15.218546 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:15.219031 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:15.219233 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:15.220473 (+ 1240us) server_negotiation.cc:299] Negotiation successful
0811 00:26:15.220859 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1371,"thread_start_us":1078,"threads_started":1}
I20260811 00:26:15.222167 6454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:15.193576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:41027 (local address 127.2.120.2:51517)
0811 00:26:15.196215 (+ 2639us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:15.196334 (+ 119us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:15.200076 (+ 3742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:15.201532 (+ 1456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:15.201546 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:15.202288 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:15.203039 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:15.203068 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:15.204297 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:15.204301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:15.205926 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:15.205942 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:15.207070 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:15.207081 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:15.207470 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:15.208603 (+ 1133us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:15.208645 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:15.211785 (+ 3140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:15.216927 (+ 5142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:15.216939 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:15.216958 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:15.217555 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:15.218138 (+ 583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:15.218142 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:15.218144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:15.218369 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:15.219251 (+ 882us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:15.219259 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:15.219699 (+ 440us) client_negotiation.cc:769] Sending connection context
0811 00:26:15.220052 (+ 353us) client_negotiation.cc:240] Negotiation successful
0811 00:26:15.220696 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1513,"thread_start_us":850,"threads_started":1}
WARNING: no policy specified for kudu/127.2.120.3@KRBTEST.COM; defaulting to no policy
I20260811 00:26:15.224727 6451 heartbeater.cc:347] Connected to a master server at 127.2.120.62:41027
I20260811 00:26:15.225128 6451 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:15.226133 6451 heartbeater.cc:511] Master 127.2.120.62:41027 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.3@KRBTEST.COM" created.
I20260811 00:26:15.228266 6080 ts_manager.cc:194] Registered new tserver with Master: 027c15e08fee4eb5bf295aed05618116 (127.2.120.2:34293)
I20260811 00:26:15.229868 6080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:51517
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
I20260811 00:26:15.261751 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:41027
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:38691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:26:15.560101 6299 heartbeater.cc:503] Master 127.2.120.62:41027 was elected leader, sending a full tablet report...
W20260811 00:26:15.674052 6458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:15.674396 6458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:15.674499 6458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:15.682240 6458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:15.682389 6458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:15.682514 6458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:26:15.694458 6458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:38691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:41027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:15.696137 6458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:15.697935 6458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:15.715696 6465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:15.715766 6467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:15.715791 6463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:15.717427 6458 server_base.cc:1034] running on GCE node
I20260811 00:26:15.718297 6458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:15.719842 6458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:15.721522 6458 hybrid_clock.cc:648] HybridClock initialized: now 1786407975721424 us; error 78 us; skew 500 ppm
I20260811 00:26:15.722101 6458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:15 dist-test-slave-fd7l krb5kdc[6030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407975, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:15.726356 6458 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:26:15.728364 6458 webserver.cc:460] Webserver started at http://127.2.120.3:44021/ using document root <none> and password file <none>
I20260811 00:26:15.729279 6458 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:15.729406 6458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:15.729815 6458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:15.733479 6458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "3dcab5f8c1204e6d997fe140d4349113"
format_stamp: "Formatted at 2026-08-11 00:26:15 on dist-test-slave-fd7l"
server_key: "a0406dc9438e8486827a71c1031cc406"
server_key_iv: "1f5da23e2c2315dfdde334877993fb9b"
server_key_version: "encryptionkey@0"
I20260811 00:26:15.734499 6458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "3dcab5f8c1204e6d997fe140d4349113"
format_stamp: "Formatted at 2026-08-11 00:26:15 on dist-test-slave-fd7l"
server_key: "a0406dc9438e8486827a71c1031cc406"
server_key_iv: "1f5da23e2c2315dfdde334877993fb9b"
server_key_version: "encryptionkey@0"
I20260811 00:26:15.741624 6458 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260811 00:26:15.746490 6474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:15.749179 6458 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:26:15.749451 6458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "3dcab5f8c1204e6d997fe140d4349113"
format_stamp: "Formatted at 2026-08-11 00:26:15 on dist-test-slave-fd7l"
server_key: "a0406dc9438e8486827a71c1031cc406"
server_key_iv: "1f5da23e2c2315dfdde334877993fb9b"
server_key_version: "encryptionkey@0"
I20260811 00:26:15.749730 6458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:15.792696 6458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:15.799245 6458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:15.799618 6458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:15.800928 6458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:15.802990 6458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:15.803195 6458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:15.803465 6458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:15.803555 6458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:15.849051 6458 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:39393
I20260811 00:26:15.849126 6603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:39393 every 8 connection(s)
I20260811 00:26:15.851222 6458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:26:15.853799 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 6458
I20260811 00:26:15.854063 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:26:15.854775 2528 external_mini_cluster.cc:1428] Setting key 8a6a47e369a4aeaca8505beb2936ee2c
Aug 11 00:26:15 dist-test-slave-fd7l krb5kdc[6030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407975, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:15.881287 6606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:15.855803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:55425 (local address 127.2.120.62:41027)
0811 00:26:15.856678 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:15.856686 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:15.857688 (+ 1002us) server_negotiation.cc:407] Connection header received
0811 00:26:15.859889 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:15.859894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:15.859977 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:15.860095 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:15.863399 (+ 3304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:15.864401 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:15.867055 (+ 2654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:15.867649 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:15.867839 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:15.873129 (+ 5290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:15.873135 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:15.873139 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:15.873182 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:15.876342 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:15.877689 (+ 1347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:15.877699 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:15.877702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:15.877982 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:15.878934 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:15.878941 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:15.878944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:15.879366 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:15.879569 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:15.880711 (+ 1142us) server_negotiation.cc:299] Negotiation successful
0811 00:26:15.880952 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":513,"threads_started":1}
I20260811 00:26:15.882582 6607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:15.856394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:41027 (local address 127.2.120.3:55425)
0811 00:26:15.857490 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:15.857546 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:15.859509 (+ 1963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:15.860927 (+ 1418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:15.860943 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:15.861897 (+ 954us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:15.862949 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:15.862985 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:15.864778 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:15.864784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:15.866671 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:15.866702 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:15.868115 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:15.868124 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:15.868483 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:15.869549 (+ 1066us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:15.869580 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:15.872781 (+ 3201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:15.876711 (+ 3930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:15.876722 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:15.876742 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:15.877361 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:15.878276 (+ 915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:15.878282 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:15.878285 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:15.878674 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:15.879617 (+ 943us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:15.879627 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:15.880105 (+ 478us) client_negotiation.cc:769] Sending connection context
0811 00:26:15.880428 (+ 323us) client_negotiation.cc:240] Negotiation successful
0811 00:26:15.881139 (+ 711us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":593,"threads_started":1}
I20260811 00:26:15.885422 6604 heartbeater.cc:347] Connected to a master server at 127.2.120.62:41027
I20260811 00:26:15.885928 6604 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:15.887353 6604 heartbeater.cc:511] Master 127.2.120.62:41027 requested a full tablet report, sending...
I20260811 00:26:15.889652 6080 ts_manager.cc:194] Registered new tserver with Master: 3dcab5f8c1204e6d997fe140d4349113 (127.2.120.3:39393)
I20260811 00:26:15.890650 6080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:55425
I20260811 00:26:15.898672 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 00:26:15 dist-test-slave-fd7l krb5kdc[6030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407975, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 00:26:15 dist-test-slave-fd7l krb5kdc[6030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407975, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:15.968107 6606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:15.947452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41018 (local address 127.2.120.62:41027)
0811 00:26:15.947796 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:15.947804 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:15.948165 (+ 361us) server_negotiation.cc:407] Connection header received
0811 00:26:15.948420 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:15.948431 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:15.948649 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:15.948779 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:15.950288 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:15.951210 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:15.952713 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:15.953148 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:15.953324 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:15.959137 (+ 5813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:15.959144 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:15.959148 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:15.959198 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:15.963418 (+ 4220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:15.964951 (+ 1533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:15.964960 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:15.964963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:15.965313 (+ 350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:15.966079 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:15.966088 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:15.966112 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:15.966677 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:15.966950 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:15.967575 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 00:26:15.967848 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
W20260811 00:26:15.969340 6080 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41018
I20260811 00:26:15.972132 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 6149
Aug 11 00:26:15 dist-test-slave-fd7l krb5kdc[6030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407975, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:26:15.997538 6618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:15.974056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35726 (local address 127.2.120.1:46361)
0811 00:26:15.974786 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:15.974794 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:15.974826 (+ 32us) server_negotiation.cc:407] Connection header received
0811 00:26:15.974903 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:15.974911 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:15.975170 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:15.975334 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:15.977295 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:15.978420 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:15.979861 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:15.980710 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:15.980849 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:15.986965 (+ 6116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:15.987013 (+ 48us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:15.987043 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:15.987148 (+ 105us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:15.991554 (+ 4406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:15.992761 (+ 1207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:15.992771 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:15.992774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:15.992965 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:15.994016 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:15.994024 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:15.994026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:15.994917 (+ 891us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:15.995366 (+ 449us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:15.996444 (+ 1078us) server_negotiation.cc:299] Negotiation successful
0811 00:26:15.997054 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":375,"threads_started":1}
W20260811 00:26:16.000319 6293 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35726
W20260811 00:26:16.002394 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 6149: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:26:16.002779 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 6149
I20260811 00:26:16.026103 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 6306
Aug 11 00:26:16 dist-test-slave-fd7l krb5kdc[6030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407975, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.120.2@KRBTEST.COM
I20260811 00:26:16.050076 6619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:16.027602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48990 (local address 127.2.120.2:34293)
0811 00:26:16.028255 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:16.028263 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:16.028295 (+ 32us) server_negotiation.cc:407] Connection header received
0811 00:26:16.028378 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:16.028385 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:16.028704 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:16.028872 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:16.030780 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:16.032033 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:16.033794 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:16.034488 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:16.034631 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:16.041897 (+ 7266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:16.041924 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:16.041946 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:16.042011 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:16.045595 (+ 3584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:16.046866 (+ 1271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:16.046876 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:16.046878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:16.047039 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:16.047824 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:16.047829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:16.047832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:16.048312 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:16.048768 (+ 456us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:16.049473 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 00:26:16.049730 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":382,"threads_started":1}
W20260811 00:26:16.051808 6445 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48990
W20260811 00:26:16.053498 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 6306: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:26:16.053696 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 6306
I20260811 00:26:16.077713 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 6458
Aug 11 00:26:16 dist-test-slave-fd7l krb5kdc[6030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407975, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.120.3@KRBTEST.COM
I20260811 00:26:16.105588 6620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:16.079426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45558 (local address 127.2.120.3:39393)
0811 00:26:16.080618 (+ 1192us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:16.080630 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:16.080674 (+ 44us) server_negotiation.cc:407] Connection header received
0811 00:26:16.080765 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:16.080775 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:16.081148 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:16.081385 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:16.084786 (+ 3401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:16.086326 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:16.088232 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:16.089372 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:16.089611 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:16.097045 (+ 7434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:16.097068 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:16.097090 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:16.097151 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:16.101044 (+ 3893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:16.102142 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:16.102151 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:16.102154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:16.102325 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:16.102822 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:16.102827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:16.102830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:16.103601 (+ 771us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:16.104036 (+ 435us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:16.104829 (+ 793us) server_negotiation.cc:299] Negotiation successful
0811 00:26:16.105235 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":912,"thread_start_us":584,"threads_started":1}
W20260811 00:26:16.107868 6599 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45558
W20260811 00:26:16.110342 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 6458: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:26:16.110637 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 6458
I20260811 00:26:16.134112 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 6044
I20260811 00:26:16.135928 6114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:16.266963 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 6044
2026-08-11T00:26:16Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4524 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:26:16 dist-test-slave-fd7l krb5kdc[6625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:26:16 dist-test-slave-fd7l krb5kdc[6625](info): set up 2 sockets
Aug 11 00:26:16 dist-test-slave-fd7l krb5kdc[6625](info): commencing operation
krb5kdc: starting...
W20260811 00:26:17.550282 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:26:17 dist-test-slave-fd7l krb5kdc[6625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:26:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:26:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:26:17.816612 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:43961
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:44453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:43961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:18.197173 6639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:18.197606 6639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:18.197788 6639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:18.206401 6639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:26:18.206588 6639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:18.206683 6639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:18.206797 6639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:26:18.206898 6639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:26:18.218169 6639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:44453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:43961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:43961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:18.219821 6639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:18.221463 6639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:18.236742 6646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:18.236724 6648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:18.236593 6645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:18.237510 6639 server_base.cc:1034] running on GCE node
I20260811 00:26:18.238523 6639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:18.240267 6639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:18.241559 6639 hybrid_clock.cc:648] HybridClock initialized: now 1786407978241462 us; error 88 us; skew 500 ppm
I20260811 00:26:18.242321 6639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:18 dist-test-slave-fd7l krb5kdc[6625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407978, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:18.246342 6639 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:26:18.248296 6639 webserver.cc:460] Webserver started at http://127.2.120.62:41657/ using document root <none> and password file <none>
I20260811 00:26:18.249176 6639 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:18.249333 6639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:18.249722 6639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:18.253352 6639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "1cbbf738b652433b939ad9d6b7a71317"
format_stamp: "Formatted at 2026-08-11 00:26:18 on dist-test-slave-fd7l"
server_key: "d85fb6a0712e24d2b0470441cfba2d07"
server_key_iv: "f0eb951e99b22ff701393cd6ac8b37be"
server_key_version: "encryptionkey@0"
I20260811 00:26:18.254326 6639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "1cbbf738b652433b939ad9d6b7a71317"
format_stamp: "Formatted at 2026-08-11 00:26:18 on dist-test-slave-fd7l"
server_key: "d85fb6a0712e24d2b0470441cfba2d07"
server_key_iv: "f0eb951e99b22ff701393cd6ac8b37be"
server_key_version: "encryptionkey@0"
I20260811 00:26:18.260905 6639 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 00:26:18.265137 6657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:18.267031 6639 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:26:18.267284 6639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "1cbbf738b652433b939ad9d6b7a71317"
format_stamp: "Formatted at 2026-08-11 00:26:18 on dist-test-slave-fd7l"
server_key: "d85fb6a0712e24d2b0470441cfba2d07"
server_key_iv: "f0eb951e99b22ff701393cd6ac8b37be"
server_key_version: "encryptionkey@0"
I20260811 00:26:18.267529 6639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:18.286932 6639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:18.293236 6639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:18.293578 6639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:18.314582 6639 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:43961
I20260811 00:26:18.314625 6715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:43961 every 8 connection(s)
I20260811 00:26:18.316177 6639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:26:18.320488 6716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:18.323923 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 6639
I20260811 00:26:18.324211 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:26:18.325078 2528 external_mini_cluster.cc:1428] Setting key f2759c8a5b040ef89a6d2e6be590072d
I20260811 00:26:18.335727 6716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317: Bootstrap starting.
I20260811 00:26:18.340384 6716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:18.341989 6716 log.cc:824] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:26:18 dist-test-slave-fd7l krb5kdc[6625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:18.346264 6716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317: No bootstrap required, opened a new log
I20260811 00:26:18.353662 6719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:18.327674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38928 (local address 127.2.120.62:43961)
0811 00:26:18.328622 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:18.328635 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:18.328683 (+ 48us) server_negotiation.cc:407] Connection header received
0811 00:26:18.330485 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:18.330512 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:18.330980 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:18.331351 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:18.333683 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:18.335116 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:18.337145 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:18.337905 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:18.338087 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:18.343850 (+ 5763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:18.343862 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:18.343885 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:18.343931 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:18.348263 (+ 4332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:18.349225 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:18.349235 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:18.349251 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:18.349514 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:18.349914 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:18.349919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:18.349922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:18.350764 (+ 842us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:18.351098 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:18.351546 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 00:26:18.352061 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":416,"threads_started":1}
I20260811 00:26:18.357461 6716 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbbf738b652433b939ad9d6b7a71317" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43961 } }
I20260811 00:26:18.357836 6716 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:18.357905 6716 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cbbf738b652433b939ad9d6b7a71317, State: Initialized, Role: FOLLOWER
I20260811 00:26:18.358501 6716 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbbf738b652433b939ad9d6b7a71317" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43961 } }
I20260811 00:26:18.358716 6716 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:26:18.358821 6716 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:26:18.358949 6716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:18.361436 6716 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbbf738b652433b939ad9d6b7a71317" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43961 } }
I20260811 00:26:18.362030 6716 leader_election.cc:302] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cbbf738b652433b939ad9d6b7a71317; no voters:
I20260811 00:26:18.362721 6716 leader_election.cc:288] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:26:18.362912 6722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:18.363693 6722 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317 [term 1 LEADER]: Becoming Leader. State: Replica: 1cbbf738b652433b939ad9d6b7a71317, State: Running, Role: LEADER
I20260811 00:26:18.364249 6722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbbf738b652433b939ad9d6b7a71317" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43961 } }
I20260811 00:26:18.365104 6716 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:26:18.367707 6723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cbbf738b652433b939ad9d6b7a71317" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbbf738b652433b939ad9d6b7a71317" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43961 } } }
I20260811 00:26:18.368117 6723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317 [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:18.368558 6724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cbbf738b652433b939ad9d6b7a71317. Latest consensus state: current_term: 1 leader_uuid: "1cbbf738b652433b939ad9d6b7a71317" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cbbf738b652433b939ad9d6b7a71317" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43961 } } }
I20260811 00:26:18.368866 6724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317 [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:18.370663 6733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:26:18.377586 6733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:26:18.389340 6733 catalog_manager.cc:1347] Generated new cluster ID: c2e9e60023f5498e881ac6db8523920b
I20260811 00:26:18.389485 6733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:26:18.405925 6733 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:26:18.407418 6733 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:26:18.415652 6733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cbbf738b652433b939ad9d6b7a71317: Generated new TSK 0
I20260811 00:26:18.416909 6733 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
I20260811 00:26:18.485654 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43961
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:44453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:18.904805 6748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:18.905200 6748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:18.905381 6748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:18.914858 6748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:18.915048 6748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:18.915310 6748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:26:18.927052 6748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:44453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:18.928897 6748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:18.930529 6748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:18.947067 6756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:18.947067 6758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:18.947103 6755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:18.958078 6748 server_base.cc:1034] running on GCE node
I20260811 00:26:18.959249 6748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:18.960842 6748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:18.962121 6748 hybrid_clock.cc:648] HybridClock initialized: now 1786407978962049 us; error 66 us; skew 500 ppm
I20260811 00:26:18.962594 6748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:18 dist-test-slave-fd7l krb5kdc[6625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407978, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:18.970666 6748 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:26:18.972878 6748 webserver.cc:460] Webserver started at http://127.2.120.1:39997/ using document root <none> and password file <none>
I20260811 00:26:18.973927 6748 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:18.974107 6748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:18.974489 6748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:18.978091 6748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "034a966a44a046b9b9e3a8c548ca1120"
format_stamp: "Formatted at 2026-08-11 00:26:18 on dist-test-slave-fd7l"
server_key: "59846f00c3d7d017658a802b168bdd32"
server_key_iv: "cbecf6d4c7486f4dccbd409713810cf3"
server_key_version: "encryptionkey@0"
I20260811 00:26:18.979039 6748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "034a966a44a046b9b9e3a8c548ca1120"
format_stamp: "Formatted at 2026-08-11 00:26:18 on dist-test-slave-fd7l"
server_key: "59846f00c3d7d017658a802b168bdd32"
server_key_iv: "cbecf6d4c7486f4dccbd409713810cf3"
server_key_version: "encryptionkey@0"
I20260811 00:26:18.986289 6748 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260811 00:26:18.991724 6765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:18.993572 6748 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 00:26:18.993825 6748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "034a966a44a046b9b9e3a8c548ca1120"
format_stamp: "Formatted at 2026-08-11 00:26:18 on dist-test-slave-fd7l"
server_key: "59846f00c3d7d017658a802b168bdd32"
server_key_iv: "cbecf6d4c7486f4dccbd409713810cf3"
server_key_version: "encryptionkey@0"
I20260811 00:26:18.994119 6748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:19.014324 6748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:19.020933 6748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:19.021299 6748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:19.023171 6748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:19.025311 6748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:19.025460 6748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:19.025632 6748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:19.025718 6748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:19.070053 6748 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:43679
I20260811 00:26:19.070338 6889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:43679 every 8 connection(s)
I20260811 00:26:19.071910 6748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:26:19.078534 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 6748
I20260811 00:26:19.078789 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:26:19.079480 2528 external_mini_cluster.cc:1428] Setting key 73ae452ae9fdfa3d4fa0aa013ca1f718
Aug 11 00:26:19 dist-test-slave-fd7l krb5kdc[6625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407978, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:19.095052 6893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:19.075301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:47911 (local address 127.2.120.62:43961)
0811 00:26:19.075832 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:19.075837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:19.076953 (+ 1116us) server_negotiation.cc:407] Connection header received
0811 00:26:19.079106 (+ 2153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:19.079113 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:19.079218 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:19.079345 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:19.081132 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:19.081872 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:19.083633 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:19.084057 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:19.084208 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:19.088145 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:19.088150 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:19.088153 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:19.088187 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:19.091480 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:19.092387 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:19.092397 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:19.092400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:19.092601 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:19.093099 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:19.093104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:19.093107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:19.093407 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:19.093572 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:19.094382 (+ 810us) server_negotiation.cc:299] Negotiation successful
0811 00:26:19.094654 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":302,"threads_started":1}
I20260811 00:26:19.096167 6894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:19.075842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:43961 (local address 127.2.120.1:47911)
0811 00:26:19.076760 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:19.076811 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:19.078754 (+ 1943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:19.079649 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:19.079659 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:19.080228 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:19.080947 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:19.080969 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:19.082090 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:19.082094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:19.083467 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:19.083483 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:19.084320 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:19.084329 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:19.084684 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:19.085459 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:19.085485 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:19.087871 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:19.091714 (+ 3843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:19.091723 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:19.091740 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:19.092232 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:19.092724 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:19.092731 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:19.092740 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:19.092962 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:19.093555 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:19.093565 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:19.093960 (+ 395us) client_negotiation.cc:769] Sending connection context
0811 00:26:19.094254 (+ 294us) client_negotiation.cc:240] Negotiation successful
0811 00:26:19.094883 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":351,"threads_started":1}
I20260811 00:26:19.098364 6890 heartbeater.cc:347] Connected to a master server at 127.2.120.62:43961
I20260811 00:26:19.098773 6890 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:19.100638 6890 heartbeater.cc:511] Master 127.2.120.62:43961 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.2.120.2@KRBTEST.COM; defaulting to no policy
I20260811 00:26:19.104496 6677 ts_manager.cc:194] Registered new tserver with Master: 034a966a44a046b9b9e3a8c548ca1120 (127.2.120.1:43679)
I20260811 00:26:19.106554 6677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:47911
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
I20260811 00:26:19.140791 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43961
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:44453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:19.540179 6897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:19.540714 6897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:19.540947 6897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:19.549794 6897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:19.550004 6897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:19.550207 6897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:26:19.562201 6897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:44453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:19.563866 6897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:19.565734 6897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:19.582826 6904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:19.582738 6906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:19.582798 6903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:19.587067 6897 server_base.cc:1034] running on GCE node
I20260811 00:26:19.588279 6897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:19.589928 6897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:19.591307 6897 hybrid_clock.cc:648] HybridClock initialized: now 1786407979591225 us; error 65 us; skew 500 ppm
I20260811 00:26:19.591914 6897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:19 dist-test-slave-fd7l krb5kdc[6625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407979, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:19.598848 6897 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:26:19.604259 6897 webserver.cc:460] Webserver started at http://127.2.120.2:35763/ using document root <none> and password file <none>
I20260811 00:26:19.605468 6897 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:19.605636 6897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:19.606065 6897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:19.609664 6897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "7296a78af8204ed78b292e7a46b8d0c1"
format_stamp: "Formatted at 2026-08-11 00:26:19 on dist-test-slave-fd7l"
server_key: "6365ba71abd7b64abc01b4f87e6a85d5"
server_key_iv: "225cf2bdeceae80358ae1ed6a77b5952"
server_key_version: "encryptionkey@0"
I20260811 00:26:19.610714 6897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "7296a78af8204ed78b292e7a46b8d0c1"
format_stamp: "Formatted at 2026-08-11 00:26:19 on dist-test-slave-fd7l"
server_key: "6365ba71abd7b64abc01b4f87e6a85d5"
server_key_iv: "225cf2bdeceae80358ae1ed6a77b5952"
server_key_version: "encryptionkey@0"
I20260811 00:26:19.617391 6897 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 00:26:19.623242 6913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:19.625643 6897 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.003s
I20260811 00:26:19.625936 6897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "7296a78af8204ed78b292e7a46b8d0c1"
format_stamp: "Formatted at 2026-08-11 00:26:19 on dist-test-slave-fd7l"
server_key: "6365ba71abd7b64abc01b4f87e6a85d5"
server_key_iv: "225cf2bdeceae80358ae1ed6a77b5952"
server_key_version: "encryptionkey@0"
I20260811 00:26:19.626261 6897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:19.643435 6897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:19.648986 6897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:19.649346 6897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:19.650656 6897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:19.652393 6897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:19.652591 6897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 00:26:19.652774 6897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:19.652858 6897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:19.696702 6897 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:45641
I20260811 00:26:19.696703 7041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:45641 every 8 connection(s)
I20260811 00:26:19.698429 6897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:26:19.704371 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 6897
I20260811 00:26:19.704651 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:26:19.705204 2528 external_mini_cluster.cc:1428] Setting key 494f905b81fd9c60962b9ed25440afff
Aug 11 00:26:19 dist-test-slave-fd7l krb5kdc[6625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407979, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:19.730284 7046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:19.702123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:49327 (local address 127.2.120.62:43961)
0811 00:26:19.702801 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:19.702811 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:19.703977 (+ 1166us) server_negotiation.cc:407] Connection header received
0811 00:26:19.706707 (+ 2730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:19.706713 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:19.706792 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:19.706921 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:19.710100 (+ 3179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:19.710892 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:19.715371 (+ 4479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:19.715930 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:19.716100 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:19.721759 (+ 5659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:19.721765 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:19.721768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:19.721812 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:19.725645 (+ 3833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:19.726844 (+ 1199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:19.726849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:19.726851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:19.727006 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:19.727546 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:19.727550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:19.727552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:19.727871 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:19.728048 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:19.729288 (+ 1240us) server_negotiation.cc:299] Negotiation successful
0811 00:26:19.729691 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":247,"threads_started":1}
I20260811 00:26:19.730911 7047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:19.702600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:43961 (local address 127.2.120.2:49327)
0811 00:26:19.703715 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:19.703759 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:19.706341 (+ 2582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:19.707225 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:19.707235 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:19.708410 (+ 1175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:19.709574 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:19.709616 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:19.711219 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:19.711231 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:19.715024 (+ 3793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:19.715068 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:19.716281 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:19.716293 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:19.716810 (+ 517us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:19.718184 (+ 1374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:19.718227 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:19.721393 (+ 3166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:19.725893 (+ 4500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:19.725912 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:19.725931 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:19.726652 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:19.727144 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:19.727150 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:19.727154 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:19.727397 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:19.728022 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:19.728032 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:19.728477 (+ 445us) client_negotiation.cc:769] Sending connection context
0811 00:26:19.728922 (+ 445us) client_negotiation.cc:240] Negotiation successful
0811 00:26:19.729425 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":834,"thread_start_us":565,"threads_started":1}
I20260811 00:26:19.733122 7042 heartbeater.cc:347] Connected to a master server at 127.2.120.62:43961
I20260811 00:26:19.733503 7042 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:19.734407 7042 heartbeater.cc:511] Master 127.2.120.62:43961 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.2.120.3@KRBTEST.COM; defaulting to no policy
I20260811 00:26:19.736337 6677 ts_manager.cc:194] Registered new tserver with Master: 7296a78af8204ed78b292e7a46b8d0c1 (127.2.120.2:45641)
I20260811 00:26:19.737260 6677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:49327
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
I20260811 00:26:19.772913 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43961
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:44453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:26:20.111056 6890 heartbeater.cc:503] Master 127.2.120.62:43961 was elected leader, sending a full tablet report...
W20260811 00:26:20.156806 7050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:20.157127 7050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:20.157298 7050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:20.165796 7050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:20.165939 7050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:20.166164 7050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:26:20.177359 7050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:44453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:20.178866 7050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:20.180312 7050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:20.197039 7057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:20.197127 7055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:20.197074 7059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:20.198271 7050 server_base.cc:1034] running on GCE node
I20260811 00:26:20.199033 7050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:20.200486 7050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:20.201999 7050 hybrid_clock.cc:648] HybridClock initialized: now 1786407980201877 us; error 120 us; skew 500 ppm
I20260811 00:26:20.202483 7050 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:20 dist-test-slave-fd7l krb5kdc[6625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407980, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:20.207465 7050 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:26:20.209578 7050 webserver.cc:460] Webserver started at http://127.2.120.3:33037/ using document root <none> and password file <none>
I20260811 00:26:20.210433 7050 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:20.210630 7050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:20.211011 7050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:20.214614 7050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "a2670965d7f24323a61f830935417ba2"
format_stamp: "Formatted at 2026-08-11 00:26:20 on dist-test-slave-fd7l"
server_key: "d4908b1c466ea573e536ec280f934bb8"
server_key_iv: "3f01e0c47aa338fd30928b9a2853a262"
server_key_version: "encryptionkey@0"
I20260811 00:26:20.215593 7050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "a2670965d7f24323a61f830935417ba2"
format_stamp: "Formatted at 2026-08-11 00:26:20 on dist-test-slave-fd7l"
server_key: "d4908b1c466ea573e536ec280f934bb8"
server_key_iv: "3f01e0c47aa338fd30928b9a2853a262"
server_key_version: "encryptionkey@0"
I20260811 00:26:20.223235 7050 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 00:26:20.229202 7067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:20.231446 7050 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:26:20.231743 7050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "a2670965d7f24323a61f830935417ba2"
format_stamp: "Formatted at 2026-08-11 00:26:20 on dist-test-slave-fd7l"
server_key: "d4908b1c466ea573e536ec280f934bb8"
server_key_iv: "3f01e0c47aa338fd30928b9a2853a262"
server_key_version: "encryptionkey@0"
I20260811 00:26:20.232013 7050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:20.254500 7050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:20.260085 7050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:20.260378 7050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:20.261723 7050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:20.263643 7050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:20.263741 7050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:20.263887 7050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:20.263963 7050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:20.311461 7050 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:37763
I20260811 00:26:20.311507 7193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:37763 every 8 connection(s)
I20260811 00:26:20.312935 7050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:26:20.321741 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 7050
I20260811 00:26:20.322017 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:26:20.322677 2528 external_mini_cluster.cc:1428] Setting key febaa1366c448f59cf1cc60225b96192
Aug 11 00:26:20 dist-test-slave-fd7l krb5kdc[6625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407980, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:20.341136 7197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:20.315983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:46441 (local address 127.2.120.62:43961)
0811 00:26:20.316491 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:20.316495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:20.317906 (+ 1411us) server_negotiation.cc:407] Connection header received
0811 00:26:20.319903 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:20.319909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:20.320008 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:20.320157 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:20.323307 (+ 3150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:20.324562 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:20.326419 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:20.327082 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:20.327256 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:20.332650 (+ 5394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:20.332658 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:20.332662 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:20.332715 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:20.336382 (+ 3667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:20.338207 (+ 1825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:20.338213 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:20.338216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:20.338421 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:20.339019 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:20.339023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:20.339025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:20.339327 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:20.339551 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:20.340366 (+ 815us) server_negotiation.cc:299] Negotiation successful
0811 00:26:20.340704 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":264,"threads_started":1}
I20260811 00:26:20.342157 7198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:20.316577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:43961 (local address 127.2.120.3:46441)
0811 00:26:20.317705 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:20.317760 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:20.319576 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:20.320449 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:20.320459 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:20.321969 (+ 1510us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:20.323043 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:20.323080 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:20.324776 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:20.324783 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:20.326249 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:20.326265 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:20.327374 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:20.327382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:20.327643 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:20.328439 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:20.328476 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:20.332167 (+ 3691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:20.336667 (+ 4500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:20.336680 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:20.336840 (+ 160us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:20.337989 (+ 1149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:20.338560 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:20.338565 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:20.338569 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:20.338874 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:20.339516 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:20.339525 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:20.339954 (+ 429us) client_negotiation.cc:769] Sending connection context
0811 00:26:20.340257 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 00:26:20.340908 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"thread_start_us":589,"threads_started":1}
I20260811 00:26:20.343948 7194 heartbeater.cc:347] Connected to a master server at 127.2.120.62:43961
I20260811 00:26:20.344494 7194 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:20.345696 7194 heartbeater.cc:511] Master 127.2.120.62:43961 requested a full tablet report, sending...
I20260811 00:26:20.348053 6677 ts_manager.cc:194] Registered new tserver with Master: a2670965d7f24323a61f830935417ba2 (127.2.120.3:37763)
I20260811 00:26:20.349362 6677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:46441
I20260811 00:26:20.353382 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 00:26:20 dist-test-slave-fd7l krb5kdc[6625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 00:26:20 dist-test-slave-fd7l krb5kdc[6625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:26:20.424062 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:20.403120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51222 (local address 127.2.120.1:43679)
0811 00:26:20.404010 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:20.404018 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:20.404057 (+ 39us) server_negotiation.cc:407] Connection header received
0811 00:26:20.404137 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:20.404145 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:20.404404 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:20.404760 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:20.406837 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:20.408128 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:20.410307 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:20.411035 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:20.411164 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:20.416335 (+ 5171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:20.416355 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:20.416375 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:20.416427 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:20.420092 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:20.421251 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:20.421261 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:20.421263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:20.421479 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:20.422017 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:20.422021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:20.422023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:20.422484 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:20.422871 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:20.423264 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 00:26:20.423604 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":466,"threads_started":1}
Aug 11 00:26:20 dist-test-slave-fd7l krb5kdc[6625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:20.446702 7197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:20.428974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38934 (local address 127.2.120.62:43961)
0811 00:26:20.429223 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:20.429230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:20.429446 (+ 216us) server_negotiation.cc:407] Connection header received
0811 00:26:20.429671 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:20.429678 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:20.429759 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:20.429866 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:20.431303 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:20.432133 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:20.433478 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:20.433911 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:20.434089 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:20.439694 (+ 5605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:20.439701 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:20.439704 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:20.439760 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:20.442820 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:20.443808 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:20.443812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:20.443815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:20.443970 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:20.444470 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:20.444474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:20.444476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:20.445138 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:20.445443 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:20.446082 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 00:26:20.446349 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260811 00:26:20.447939 6677 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38934
I20260811 00:26:20.449690 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 6748
I20260811 00:26:20.472810 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:20.450790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51236 (local address 127.2.120.1:43679)
0811 00:26:20.451024 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:20.451033 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:20.451431 (+ 398us) server_negotiation.cc:407] Connection header received
0811 00:26:20.451591 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:20.451598 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:20.451698 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:20.451812 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:20.454058 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:20.455322 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:20.458364 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:20.459245 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:20.459442 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:20.463752 (+ 4310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:20.463761 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:20.463770 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:20.463839 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:20.468463 (+ 4624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:20.469778 (+ 1315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:20.469784 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:20.469788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:20.470000 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:20.470544 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:20.470552 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:20.470555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:20.470968 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:20.471282 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:20.472064 (+ 782us) server_negotiation.cc:299] Negotiation successful
0811 00:26:20.472313 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 00:26:20.473661 6884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:20.586449 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 6748
I20260811 00:26:20.613202 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 6897
Aug 11 00:26:20 dist-test-slave-fd7l krb5kdc[6625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.2@KRBTEST.COM
I20260811 00:26:20.634706 7210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:20.614558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47662 (local address 127.2.120.2:45641)
0811 00:26:20.615347 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:20.615358 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:20.615396 (+ 38us) server_negotiation.cc:407] Connection header received
0811 00:26:20.615480 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:20.615488 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:20.615811 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:20.615994 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:20.617730 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:20.618886 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:20.620312 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:20.621193 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:20.621362 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:20.626300 (+ 4938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:20.626317 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:20.626336 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:20.626384 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:20.629822 (+ 3438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:20.631115 (+ 1293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:20.631124 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:20.631126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:20.631295 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:20.632067 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:20.632073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:20.632076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:20.632875 (+ 799us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:20.633228 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:20.633765 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 00:26:20.634065 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":402,"threads_started":1}
I20260811 00:26:20.636474 7036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:20.731000 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 6897
I20260811 00:26:20.753393 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 7050
Aug 11 00:26:20 dist-test-slave-fd7l krb5kdc[6625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.3@KRBTEST.COM
I20260811 00:26:20.775561 7212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:20.755025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53898 (local address 127.2.120.3:37763)
0811 00:26:20.755751 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:20.755760 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:20.755799 (+ 39us) server_negotiation.cc:407] Connection header received
0811 00:26:20.755917 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:20.755925 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:20.756231 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:20.756790 (+ 559us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:20.758399 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:20.759527 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:20.761504 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:20.762212 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:20.762412 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:20.767919 (+ 5507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:20.767939 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:20.767960 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:20.768012 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:20.772008 (+ 3996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:20.773178 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:20.773188 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:20.773191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:20.773413 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:20.773861 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:20.773865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:20.773867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:20.774298 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:20.774543 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:20.774876 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 00:26:20.775114 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":361,"threads_started":1}
I20260811 00:26:20.777267 7188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:20.866011 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 7050
I20260811 00:26:20.887780 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 6639
I20260811 00:26:20.889323 6710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:20.996973 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 6639
2026-08-11T00:26:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4727 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:26:21 dist-test-slave-fd7l krb5kdc[7218](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:26:21 dist-test-slave-fd7l krb5kdc[7218](info): set up 2 sockets
Aug 11 00:26:21 dist-test-slave-fd7l krb5kdc[7218](info): commencing operation
krb5kdc: starting...
W20260811 00:26:22.246541 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.007s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:26:22 dist-test-slave-fd7l krb5kdc[7218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:26:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:26:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:26:22.525084 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:41125
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:41125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:22.935110 7233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:22.935561 7233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:22.935750 7233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:22.945312 7233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:26:22.945626 7233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:22.945749 7233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:22.945858 7233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:26:22.945966 7233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:26:22.957326 7233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:41125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:41125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:22.958904 7233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:22.960366 7233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:22.973839 7239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:22.974066 7240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:22.973784 7243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:22.975497 7233 server_base.cc:1034] running on GCE node
I20260811 00:26:22.976701 7233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:22.978469 7233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:22.979921 7233 hybrid_clock.cc:648] HybridClock initialized: now 1786407982979713 us; error 75 us; skew 500 ppm
I20260811 00:26:22.980756 7233 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:22 dist-test-slave-fd7l krb5kdc[7218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407982, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:22.984922 7233 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:26:22.986660 7233 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:22.986835 7233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:22.987224 7233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:22.990779 7233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "e103360b8d4f49dc90ce0a8d25899d1a"
format_stamp: "Formatted at 2026-08-11 00:26:22 on dist-test-slave-fd7l"
server_key: "d2c6a83f9e45f2dabc0822b425a4d8c1"
server_key_iv: "287626747b5a71bf3a9d4de78690baf2"
server_key_version: "encryptionkey@0"
I20260811 00:26:22.991776 7233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "e103360b8d4f49dc90ce0a8d25899d1a"
format_stamp: "Formatted at 2026-08-11 00:26:22 on dist-test-slave-fd7l"
server_key: "d2c6a83f9e45f2dabc0822b425a4d8c1"
server_key_iv: "287626747b5a71bf3a9d4de78690baf2"
server_key_version: "encryptionkey@0"
I20260811 00:26:22.998127 7233 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 00:26:23.002926 7250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:23.004721 7233 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 00:26:23.004967 7233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "e103360b8d4f49dc90ce0a8d25899d1a"
format_stamp: "Formatted at 2026-08-11 00:26:22 on dist-test-slave-fd7l"
server_key: "d2c6a83f9e45f2dabc0822b425a4d8c1"
server_key_iv: "287626747b5a71bf3a9d4de78690baf2"
server_key_version: "encryptionkey@0"
I20260811 00:26:23.005247 7233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:23.022418 7233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:23.027894 7233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:23.028245 7233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:23.051263 7233 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:41125
I20260811 00:26:23.051287 7305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:41125 every 8 connection(s)
I20260811 00:26:23.052325 7233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:26:23.055217 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 7233
I20260811 00:26:23.055517 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:26:23.056180 2528 external_mini_cluster.cc:1428] Setting key f8ec8215b46fd8f09622089e0f8ef2eb
I20260811 00:26:23.056964 7306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:23.072769 7306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a: Bootstrap starting.
Aug 11 00:26:23 dist-test-slave-fd7l krb5kdc[7218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:23.077410 7306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:23.079080 7306 log.cc:824] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:23.084192 7306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a: No bootstrap required, opened a new log
I20260811 00:26:23.086524 7309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:23.058536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39382 (local address 127.2.120.62:41125)
0811 00:26:23.059439 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:23.059452 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:23.059496 (+ 44us) server_negotiation.cc:407] Connection header received
0811 00:26:23.062055 (+ 2559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:23.062087 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:23.062625 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:23.063032 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:23.065282 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:23.067017 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:23.069225 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:23.070055 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:23.070221 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:23.075628 (+ 5407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:23.075640 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:23.075664 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:23.075727 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:23.080226 (+ 4499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:23.081467 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:23.081482 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:23.081498 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:23.081981 (+ 483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:23.082591 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:23.082598 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:23.082601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:23.083461 (+ 860us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:23.083932 (+ 471us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:23.084431 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 00:26:23.085009 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":442,"threads_started":1}
I20260811 00:26:23.094201 7306 raft_consensus.cc:348] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e103360b8d4f49dc90ce0a8d25899d1a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 41125 } }
I20260811 00:26:23.094635 7306 raft_consensus.cc:374] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:23.094713 7306 raft_consensus.cc:729] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e103360b8d4f49dc90ce0a8d25899d1a, State: Initialized, Role: FOLLOWER
I20260811 00:26:23.095292 7306 consensus_queue.cc:260] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e103360b8d4f49dc90ce0a8d25899d1a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 41125 } }
I20260811 00:26:23.095603 7306 raft_consensus.cc:388] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:26:23.095718 7306 raft_consensus.cc:482] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:26:23.095904 7306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:23.098948 7306 raft_consensus.cc:504] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e103360b8d4f49dc90ce0a8d25899d1a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 41125 } }
I20260811 00:26:23.099900 7306 leader_election.cc:302] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e103360b8d4f49dc90ce0a8d25899d1a; no voters:
I20260811 00:26:23.100847 7306 leader_election.cc:288] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:26:23.101052 7311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:23.102039 7311 raft_consensus.cc:686] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a [term 1 LEADER]: Becoming Leader. State: Replica: e103360b8d4f49dc90ce0a8d25899d1a, State: Running, Role: LEADER
I20260811 00:26:23.102813 7311 consensus_queue.cc:237] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e103360b8d4f49dc90ce0a8d25899d1a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 41125 } }
I20260811 00:26:23.103883 7306 sys_catalog.cc:564] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:26:23.106707 7315 sys_catalog.cc:455] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e103360b8d4f49dc90ce0a8d25899d1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e103360b8d4f49dc90ce0a8d25899d1a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 41125 } } }
I20260811 00:26:23.107110 7316 sys_catalog.cc:455] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e103360b8d4f49dc90ce0a8d25899d1a. Latest consensus state: current_term: 1 leader_uuid: "e103360b8d4f49dc90ce0a8d25899d1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e103360b8d4f49dc90ce0a8d25899d1a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 41125 } } }
I20260811 00:26:23.107498 7316 sys_catalog.cc:458] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:23.107497 7315 sys_catalog.cc:458] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:23.121616 7322 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:26:23.127962 7322 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:26:23.138536 7322 catalog_manager.cc:1347] Generated new cluster ID: 334be7ded02540e9a3aea7d3d65eeceb
I20260811 00:26:23.138712 7322 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:26:23.155088 7322 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:26:23.157145 7322 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:26:23.180254 7322 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a: Generated new TSK 0
I20260811 00:26:23.181401 7322 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
I20260811 00:26:23.249626 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:41125
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:23.662551 7335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:23.662879 7335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:23.663029 7335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:23.670845 7335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:23.670989 7335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:23.671191 7335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:26:23.682102 7335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:41125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:23.683805 7335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:23.686434 7335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:23.704257 7341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:23.704257 7340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:23.704276 7343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:23.705051 7335 server_base.cc:1034] running on GCE node
I20260811 00:26:23.706070 7335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:23.707270 7335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:23.708616 7335 hybrid_clock.cc:648] HybridClock initialized: now 1786407983708513 us; error 72 us; skew 500 ppm
I20260811 00:26:23.709089 7335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:23 dist-test-slave-fd7l krb5kdc[7218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407983, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:23.713809 7335 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:26:23.715508 7335 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:23.715694 7335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:23.716078 7335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:23.719719 7335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "f64fe381c580438bae27a024f94afcc7"
format_stamp: "Formatted at 2026-08-11 00:26:23 on dist-test-slave-fd7l"
server_key: "7dbd1faf0bee3746f3e2799549fbf752"
server_key_iv: "af13b839dfc004299284d2f8405f2654"
server_key_version: "encryptionkey@0"
I20260811 00:26:23.720780 7335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "f64fe381c580438bae27a024f94afcc7"
format_stamp: "Formatted at 2026-08-11 00:26:23 on dist-test-slave-fd7l"
server_key: "7dbd1faf0bee3746f3e2799549fbf752"
server_key_iv: "af13b839dfc004299284d2f8405f2654"
server_key_version: "encryptionkey@0"
I20260811 00:26:23.727243 7335 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260811 00:26:23.733196 7349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:23.735320 7335 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260811 00:26:23.735623 7335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "f64fe381c580438bae27a024f94afcc7"
format_stamp: "Formatted at 2026-08-11 00:26:23 on dist-test-slave-fd7l"
server_key: "7dbd1faf0bee3746f3e2799549fbf752"
server_key_iv: "af13b839dfc004299284d2f8405f2654"
server_key_version: "encryptionkey@0"
I20260811 00:26:23.735932 7335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:23.757154 7335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:23.762789 7335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:23.763156 7335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:23.764510 7335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:23.766153 7335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:23.766312 7335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:23.766469 7335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:23.766571 7335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:23.806488 7335 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:37757
I20260811 00:26:23.806562 7473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:37757 every 8 connection(s)
I20260811 00:26:23.807492 7335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:26:23.810045 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 7335
I20260811 00:26:23.810338 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:26:23.811179 2528 external_mini_cluster.cc:1428] Setting key 5797358521c41d6cd9c853bf63d1dd78
Aug 11 00:26:23 dist-test-slave-fd7l krb5kdc[7218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407983, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:23.837831 7476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:23.811062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:54789 (local address 127.2.120.62:41125)
0811 00:26:23.813017 (+ 1955us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:23.813038 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:23.813389 (+ 351us) server_negotiation.cc:407] Connection header received
0811 00:26:23.816387 (+ 2998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:23.816394 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:23.816604 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:23.816855 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:23.819465 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:23.820250 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:23.823356 (+ 3106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:23.824169 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:23.824360 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:23.828997 (+ 4637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:23.829008 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:23.829016 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:23.829072 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:23.833366 (+ 4294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:23.834577 (+ 1211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:23.834587 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:23.834594 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:23.834792 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:23.835356 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:23.835364 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:23.835370 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:23.835837 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:23.836016 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:23.837045 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0811 00:26:23.837353 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1796,"thread_start_us":1646,"threads_started":1}
WARNING: no policy specified for kudu/127.2.120.2@KRBTEST.COM; defaulting to no policy
I20260811 00:26:23.838639 7477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:23.812104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:41125 (local address 127.2.120.1:54789)
0811 00:26:23.813216 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:23.813264 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:23.815834 (+ 2570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:23.817208 (+ 1374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:23.817224 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:23.818175 (+ 951us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:23.819175 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:23.819209 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:23.820476 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:23.820485 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:23.823101 (+ 2616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:23.823122 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:23.824596 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:23.824615 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:23.824891 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:23.825985 (+ 1094us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:23.826014 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:23.828670 (+ 2656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:23.833647 (+ 4977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:23.833695 (+ 48us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:23.833721 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:23.834367 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:23.834961 (+ 594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:23.834966 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:23.834968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:23.835191 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:23.836033 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:23.836041 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:23.836478 (+ 437us) client_negotiation.cc:769] Sending connection context
0811 00:26:23.836858 (+ 380us) client_negotiation.cc:240] Negotiation successful
0811 00:26:23.837355 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"thread_start_us":745,"threads_started":1}
I20260811 00:26:23.840559 7474 heartbeater.cc:347] Connected to a master server at 127.2.120.62:41125
I20260811 00:26:23.840929 7474 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:23.841740 7474 heartbeater.cc:511] Master 127.2.120.62:41125 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.2@KRBTEST.COM" created.
I20260811 00:26:23.844439 7267 ts_manager.cc:194] Registered new tserver with Master: f64fe381c580438bae27a024f94afcc7 (127.2.120.1:37757)
I20260811 00:26:23.847206 7267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:54789
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
I20260811 00:26:23.873667 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:41125
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:24.300248 7482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:24.300681 7482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:24.300819 7482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:24.309352 7482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:24.309520 7482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:24.309648 7482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:26:24.320724 7482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:41125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:24.322196 7482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:24.323661 7482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:24.338313 7490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:24.338286 7493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:24.338590 7491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:24.339228 7482 server_base.cc:1034] running on GCE node
I20260811 00:26:24.340018 7482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:24.341527 7482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:24.342806 7482 hybrid_clock.cc:648] HybridClock initialized: now 1786407984342733 us; error 72 us; skew 500 ppm
I20260811 00:26:24.343257 7482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:24 dist-test-slave-fd7l krb5kdc[7218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407984, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:24.352807 7482 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:26:24.354866 7482 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:24.355361 7482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:24.356009 7482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:24.360008 7482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "1511f919ab134f45a2b48b54b9d85f36"
format_stamp: "Formatted at 2026-08-11 00:26:24 on dist-test-slave-fd7l"
server_key: "4280378c6cfacbbd980d607415374b26"
server_key_iv: "182304e40fc60403a15040d3c9b58871"
server_key_version: "encryptionkey@0"
I20260811 00:26:24.361101 7482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "1511f919ab134f45a2b48b54b9d85f36"
format_stamp: "Formatted at 2026-08-11 00:26:24 on dist-test-slave-fd7l"
server_key: "4280378c6cfacbbd980d607415374b26"
server_key_iv: "182304e40fc60403a15040d3c9b58871"
server_key_version: "encryptionkey@0"
I20260811 00:26:24.367501 7482 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:26:24.372818 7500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:24.375347 7482 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 00:26:24.375710 7482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "1511f919ab134f45a2b48b54b9d85f36"
format_stamp: "Formatted at 2026-08-11 00:26:24 on dist-test-slave-fd7l"
server_key: "4280378c6cfacbbd980d607415374b26"
server_key_iv: "182304e40fc60403a15040d3c9b58871"
server_key_version: "encryptionkey@0"
I20260811 00:26:24.376062 7482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:24.394177 7482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:24.399686 7482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:24.400041 7482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:24.401324 7482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:24.403244 7482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:24.403568 7482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:24.403879 7482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:24.404022 7482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:24.449750 7482 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:33333
I20260811 00:26:24.450040 7632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:33333 every 8 connection(s)
I20260811 00:26:24.451041 7482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:26:24.454126 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 7482
I20260811 00:26:24.454382 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:26:24.455024 2528 external_mini_cluster.cc:1428] Setting key 68aa1da646d0e197b2274a5e3f1d610c
Aug 11 00:26:24 dist-test-slave-fd7l krb5kdc[7218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407984, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:24.481091 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:24.455031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:54019 (local address 127.2.120.62:41125)
0811 00:26:24.455764 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:24.455773 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:24.457675 (+ 1902us) server_negotiation.cc:407] Connection header received
0811 00:26:24.460322 (+ 2647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:24.460333 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:24.460478 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:24.460739 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:24.463093 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:24.464452 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:24.467443 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:24.468452 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:24.468738 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:24.473384 (+ 4646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:24.473391 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:24.473395 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:24.473435 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:24.477160 (+ 3725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:24.478232 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:24.478238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:24.478242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:24.478398 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:24.478938 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:24.478946 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:24.478949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:24.479287 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:24.479492 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:24.480369 (+ 877us) server_negotiation.cc:299] Negotiation successful
0811 00:26:24.480694 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":353,"threads_started":1}
I20260811 00:26:24.482374 7636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:24.455874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:41125 (local address 127.2.120.2:54019)
0811 00:26:24.457380 (+ 1506us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:24.457455 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:24.459805 (+ 2350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:24.461141 (+ 1336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:24.461153 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:24.461850 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:24.462764 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:24.462789 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:24.464901 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:24.464909 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:24.467078 (+ 2169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:24.467117 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:24.468940 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:24.468954 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:24.469263 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:24.470180 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:24.470216 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:24.473022 (+ 2806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:24.477420 (+ 4398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:24.477434 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:24.477452 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:24.478061 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:24.478567 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:24.478571 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:24.478577 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:24.478798 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:24.479484 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:24.479494 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:24.479944 (+ 450us) client_negotiation.cc:769] Sending connection context
0811 00:26:24.480283 (+ 339us) client_negotiation.cc:240] Negotiation successful
0811 00:26:24.480966 (+ 683us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1110,"thread_start_us":791,"threads_started":1}
I20260811 00:26:24.484767 7633 heartbeater.cc:347] Connected to a master server at 127.2.120.62:41125
WARNING: no policy specified for kudu/127.2.120.3@KRBTEST.COM; defaulting to no policy
I20260811 00:26:24.485200 7633 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:24.486114 7633 heartbeater.cc:511] Master 127.2.120.62:41125 requested a full tablet report, sending...
I20260811 00:26:24.488684 7267 ts_manager.cc:194] Registered new tserver with Master: 1511f919ab134f45a2b48b54b9d85f36 (127.2.120.2:33333)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.3@KRBTEST.COM" created.
I20260811 00:26:24.489991 7267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:54019
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
I20260811 00:26:24.524945 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:41125
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:26:24.851447 7474 heartbeater.cc:503] Master 127.2.120.62:41125 was elected leader, sending a full tablet report...
W20260811 00:26:24.940790 7639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:24.941283 7639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:24.941478 7639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:24.950845 7639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:24.951023 7639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:24.951169 7639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:26:24.963774 7639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:41125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:24.965289 7639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:24.966763 7639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:24.986131 7644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:24.986142 7645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:24.986218 7647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:24.987905 7639 server_base.cc:1034] running on GCE node
I20260811 00:26:24.988914 7639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:24.990267 7639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:24.993791 7639 hybrid_clock.cc:648] HybridClock initialized: now 1786407984993657 us; error 110 us; skew 500 ppm
I20260811 00:26:24.994342 7639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:24 dist-test-slave-fd7l krb5kdc[7218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407984, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:24.998916 7639 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:26:25.000458 7639 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:25.000684 7639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:25.001070 7639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:25.004647 7639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "3f900da1efe04988aa68f68bf00a1de2"
format_stamp: "Formatted at 2026-08-11 00:26:24 on dist-test-slave-fd7l"
server_key: "28720277840d546679009f22ac5536e4"
server_key_iv: "4ec828b275e0e9d98f2d0d839be4d1e0"
server_key_version: "encryptionkey@0"
I20260811 00:26:25.005677 7639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "3f900da1efe04988aa68f68bf00a1de2"
format_stamp: "Formatted at 2026-08-11 00:26:24 on dist-test-slave-fd7l"
server_key: "28720277840d546679009f22ac5536e4"
server_key_iv: "4ec828b275e0e9d98f2d0d839be4d1e0"
server_key_version: "encryptionkey@0"
I20260811 00:26:25.012954 7639 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 00:26:25.017823 7654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:25.019945 7639 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260811 00:26:25.020239 7639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "3f900da1efe04988aa68f68bf00a1de2"
format_stamp: "Formatted at 2026-08-11 00:26:24 on dist-test-slave-fd7l"
server_key: "28720277840d546679009f22ac5536e4"
server_key_iv: "4ec828b275e0e9d98f2d0d839be4d1e0"
server_key_version: "encryptionkey@0"
I20260811 00:26:25.020627 7639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:25.039971 7639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:25.046204 7639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:25.046623 7639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:25.047946 7639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:25.049878 7639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:25.050046 7639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:25.050211 7639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:25.050321 7639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:25.094033 7639 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:36923
I20260811 00:26:25.094098 7783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:36923 every 8 connection(s)
I20260811 00:26:25.095252 7639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:26:25.096954 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 7639
I20260811 00:26:25.097189 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:26:25.097671 2528 external_mini_cluster.cc:1428] Setting key 0258285dae277e4c532ab508867f1cce
Aug 11 00:26:25 dist-test-slave-fd7l krb5kdc[7218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407984, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:25.120888 7786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:25.098603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:39993 (local address 127.2.120.62:41125)
0811 00:26:25.100945 (+ 2342us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:25.100950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:25.100971 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:26:25.103288 (+ 2317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:25.103293 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:25.103403 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:25.103561 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:25.105602 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.106402 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.108104 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.108617 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.108759 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:25.113248 (+ 4489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:25.113254 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:25.113258 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:25.113296 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:25.116870 (+ 3574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:25.117887 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:25.117909 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:25.117914 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:25.118089 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:25.118654 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:25.118658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:25.118660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:25.118983 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:25.119120 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:25.120155 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0811 00:26:25.120489 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2211,"thread_start_us":2056,"threads_started":1}
I20260811 00:26:25.121640 7787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:25.099181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:41125 (local address 127.2.120.3:39993)
0811 00:26:25.100316 (+ 1135us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:25.100405 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:25.102956 (+ 2551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:25.103866 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:25.103876 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:25.104701 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:25.105372 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:25.105393 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.106590 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.106600 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:25.107927 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:25.107943 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.108890 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.108899 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:25.109174 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:25.110198 (+ 1024us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:25.110233 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:25.112943 (+ 2710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:25.117065 (+ 4122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:25.117074 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:25.117092 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:25.117665 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:25.118259 (+ 594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:25.118264 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:25.118267 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:25.118487 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:25.119133 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:25.119140 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:25.119552 (+ 412us) client_negotiation.cc:769] Sending connection context
0811 00:26:25.119823 (+ 271us) client_negotiation.cc:240] Negotiation successful
0811 00:26:25.120310 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":411,"threads_started":1}
I20260811 00:26:25.123677 7784 heartbeater.cc:347] Connected to a master server at 127.2.120.62:41125
I20260811 00:26:25.124122 7784 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:25.125146 7784 heartbeater.cc:511] Master 127.2.120.62:41125 requested a full tablet report, sending...
I20260811 00:26:25.127463 7267 ts_manager.cc:194] Registered new tserver with Master: 3f900da1efe04988aa68f68bf00a1de2 (127.2.120.3:36923)
I20260811 00:26:25.129093 7267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:39993
I20260811 00:26:25.129778 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:26:25.149811 7786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:25.135372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56514 (local address 127.2.120.62:41125)
0811 00:26:25.135591 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:25.135600 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:25.136227 (+ 627us) server_negotiation.cc:407] Connection header received
0811 00:26:25.136346 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:25.136350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:25.136431 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:25.136601 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:25.138065 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.138885 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.140473 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.141005 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.141118 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:25.143848 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:25.143854 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:25.143858 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:25.143911 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:25.146901 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:25.147590 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:25.147594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:25.147596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:25.147727 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:25.148202 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:25.148206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:25.148209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:25.148670 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:25.148884 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:25.149233 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 00:26:25.149421 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 00:26:25.154090 7267 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:26:25.162098 7267 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:26:25.204700 7802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:25.179675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:37757 (local address 127.0.0.1:57064)
0811 00:26:25.193987 (+ 14312us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:25.194019 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:25.194207 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:25.196533 (+ 2326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:25.196538 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:25.196565 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:25.197118 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:25.197134 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.198847 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.198855 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:25.201481 (+ 2626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:25.201495 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.203569 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.203576 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:25.203827 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:25.203862 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 00:26:25.203943 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 00:26:25.204117 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14089,"mutex_wait_us":74,"thread_start_us":15947,"threads_started":1}
I20260811 00:26:25.204843 7803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:25.180538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57064 (local address 127.2.120.1:37757)
0811 00:26:25.195791 (+ 15253us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:25.195800 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:25.195835 (+ 35us) server_negotiation.cc:407] Connection header received
0811 00:26:25.195967 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:25.195975 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:25.196241 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:25.196555 (+ 314us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:25.197368 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.198710 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.201675 (+ 2965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.203295 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.203816 (+ 521us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:25.204070 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:25.204168 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 00:26:25.204378 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15052,"thread_start_us":14855,"threads_started":1}
I20260811 00:26:25.204341 7801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:25.180418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:36923 (local address 127.0.0.1:54222)
0811 00:26:25.195763 (+ 15345us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:25.195787 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:25.196037 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:25.196873 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:25.196878 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:25.196908 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:25.197516 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:25.197530 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.199650 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.199656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:25.201830 (+ 2174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:25.201841 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.203192 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.203198 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:25.203396 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:25.203458 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 00:26:25.203605 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 00:26:25.203857 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15194,"mutex_wait_us":70,"spinlock_wait_cycles":1408,"thread_start_us":13927,"threads_started":1}
I20260811 00:26:25.205797 7800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:25.179210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53402 (local address 127.2.120.2:33333)
0811 00:26:25.194651 (+ 15441us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:25.194666 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:25.194710 (+ 44us) server_negotiation.cc:407] Connection header received
0811 00:26:25.194848 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:25.194860 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:25.195214 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:25.195473 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:25.196578 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.198746 (+ 2168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.201267 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.202938 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.203108 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:25.203349 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:25.205123 (+ 1774us) server_negotiation.cc:299] Negotiation successful
0811 00:26:25.205447 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15166,"thread_start_us":15879,"threads_started":1}
I20260811 00:26:25.205821 7805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:25.181166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54222 (local address 127.2.120.3:36923)
0811 00:26:25.195191 (+ 14025us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:25.195200 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:25.195893 (+ 693us) server_negotiation.cc:407] Connection header received
0811 00:26:25.196249 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:25.196259 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:25.196475 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:25.196816 (+ 341us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:25.197772 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.199485 (+ 1713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.201987 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.202961 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.204749 (+ 1788us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:25.205339 (+ 590us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:25.205428 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 00:26:25.205574 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13832,"thread_start_us":13621,"threads_started":1}
I20260811 00:26:25.206678 7799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:25.179003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:33333 (local address 127.0.0.1:53402)
0811 00:26:25.179989 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:25.180035 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:25.180259 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:25.195808 (+ 15549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:25.195818 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:25.195853 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:25.196286 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:25.196300 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.198938 (+ 2638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.198944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:25.201088 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:25.201102 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.203534 (+ 2432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.203584 (+ 50us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:25.204472 (+ 888us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:25.204555 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 00:26:25.206057 (+ 1502us) client_negotiation.cc:240] Negotiation successful
0811 00:26:25.206243 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":36,"spinlock_wait_cycles":20608,"thread_start_us":405,"threads_started":1}
I20260811 00:26:25.210340 7711 tablet_service.cc:1467] Processing CreateTablet for tablet d30c2130e9694ef9a95f0d9262015a07 (DEFAULT_TABLE table=test-table [id=6aa1f1c554c54524bb2075b1770f3a58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:25.210529 7403 tablet_service.cc:1467] Processing CreateTablet for tablet d30c2130e9694ef9a95f0d9262015a07 (DEFAULT_TABLE table=test-table [id=6aa1f1c554c54524bb2075b1770f3a58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:25.211052 7557 tablet_service.cc:1467] Processing CreateTablet for tablet d30c2130e9694ef9a95f0d9262015a07 (DEFAULT_TABLE table=test-table [id=6aa1f1c554c54524bb2075b1770f3a58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:25.212420 7711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30c2130e9694ef9a95f0d9262015a07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:25.212745 7557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30c2130e9694ef9a95f0d9262015a07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:25.213415 7403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d30c2130e9694ef9a95f0d9262015a07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:25.227599 7806 tablet_bootstrap.cc:492] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7: Bootstrap starting.
I20260811 00:26:25.231396 7807 tablet_bootstrap.cc:492] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2: Bootstrap starting.
I20260811 00:26:25.235215 7806 tablet_bootstrap.cc:654] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:25.237030 7807 tablet_bootstrap.cc:654] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:25.237396 7809 tablet_bootstrap.cc:492] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36: Bootstrap starting.
I20260811 00:26:25.237454 7806 log.cc:824] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:25.238765 7807 log.cc:824] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:25.242493 7809 tablet_bootstrap.cc:654] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:25.242728 7806 tablet_bootstrap.cc:492] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7: No bootstrap required, opened a new log
I20260811 00:26:25.243130 7806 ts_tablet_manager.cc:1397] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.000s
I20260811 00:26:25.244138 7809 log.cc:824] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:25.246279 7807 tablet_bootstrap.cc:492] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2: No bootstrap required, opened a new log
I20260811 00:26:25.246636 7807 ts_tablet_manager.cc:1397] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20260811 00:26:25.247565 7809 tablet_bootstrap.cc:492] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36: No bootstrap required, opened a new log
I20260811 00:26:25.247874 7809 ts_tablet_manager.cc:1397] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260811 00:26:25.255059 7806 raft_consensus.cc:348] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1511f919ab134f45a2b48b54b9d85f36" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33333 } } peers { permanent_uuid: "3f900da1efe04988aa68f68bf00a1de2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36923 } } peers { permanent_uuid: "f64fe381c580438bae27a024f94afcc7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37757 } }
I20260811 00:26:25.255661 7806 raft_consensus.cc:374] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:25.255785 7806 raft_consensus.cc:729] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f64fe381c580438bae27a024f94afcc7, State: Initialized, Role: FOLLOWER
I20260811 00:26:25.256659 7806 consensus_queue.cc:260] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1511f919ab134f45a2b48b54b9d85f36" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33333 } } peers { permanent_uuid: "3f900da1efe04988aa68f68bf00a1de2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36923 } } peers { permanent_uuid: "f64fe381c580438bae27a024f94afcc7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37757 } }
I20260811 00:26:25.256774 7809 raft_consensus.cc:348] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1511f919ab134f45a2b48b54b9d85f36" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33333 } } peers { permanent_uuid: "3f900da1efe04988aa68f68bf00a1de2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36923 } } peers { permanent_uuid: "f64fe381c580438bae27a024f94afcc7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37757 } }
I20260811 00:26:25.257308 7809 raft_consensus.cc:374] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:25.257437 7809 raft_consensus.cc:729] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1511f919ab134f45a2b48b54b9d85f36, State: Initialized, Role: FOLLOWER
I20260811 00:26:25.258234 7809 consensus_queue.cc:260] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1511f919ab134f45a2b48b54b9d85f36" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33333 } } peers { permanent_uuid: "3f900da1efe04988aa68f68bf00a1de2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36923 } } peers { permanent_uuid: "f64fe381c580438bae27a024f94afcc7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37757 } }
I20260811 00:26:25.258307 7807 raft_consensus.cc:348] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1511f919ab134f45a2b48b54b9d85f36" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33333 } } peers { permanent_uuid: "3f900da1efe04988aa68f68bf00a1de2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36923 } } peers { permanent_uuid: "f64fe381c580438bae27a024f94afcc7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37757 } }
I20260811 00:26:25.258816 7807 raft_consensus.cc:374] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:25.258926 7807 raft_consensus.cc:729] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f900da1efe04988aa68f68bf00a1de2, State: Initialized, Role: FOLLOWER
I20260811 00:26:25.259001 7806 ts_tablet_manager.cc:1428] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
I20260811 00:26:25.259758 7807 consensus_queue.cc:260] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1511f919ab134f45a2b48b54b9d85f36" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33333 } } peers { permanent_uuid: "3f900da1efe04988aa68f68bf00a1de2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36923 } } peers { permanent_uuid: "f64fe381c580438bae27a024f94afcc7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37757 } }
I20260811 00:26:25.260550 7633 heartbeater.cc:503] Master 127.2.120.62:41125 was elected leader, sending a full tablet report...
I20260811 00:26:25.261073 7809 ts_tablet_manager.cc:1428] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260811 00:26:25.263447 7807 ts_tablet_manager.cc:1428] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260811 00:26:25.263590 7784 heartbeater.cc:503] Master 127.2.120.62:41125 was elected leader, sending a full tablet report...
W20260811 00:26:25.310659 7475 tablet.cc:2369] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:26:25.347220 7785 tablet.cc:2369] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:26:25.372386 7815 raft_consensus.cc:482] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:26:25.372836 7815 raft_consensus.cc:504] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1511f919ab134f45a2b48b54b9d85f36" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33333 } } peers { permanent_uuid: "3f900da1efe04988aa68f68bf00a1de2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36923 } } peers { permanent_uuid: "f64fe381c580438bae27a024f94afcc7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37757 } }
I20260811 00:26:25.374771 7815 leader_election.cc:288] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1511f919ab134f45a2b48b54b9d85f36 (127.2.120.2:33333), f64fe381c580438bae27a024f94afcc7 (127.2.120.1:37757)
I20260811 00:26:25.383126 7787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:25.375314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:33333 (local address 127.2.120.3:45241)
0811 00:26:25.375553 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:25.375576 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:25.375726 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:25.376216 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:25.376225 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:25.376277 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:25.376639 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:25.376661 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.378104 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.378109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:25.380246 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:25.380263 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.382001 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.382007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:25.382268 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:25.382313 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 00:26:25.382457 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 00:26:25.382775 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 00:26:25.383169 7800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:25.375397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:45241 (local address 127.2.120.2:33333)
0811 00:26:25.375611 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:25.375619 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:25.375642 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:26:25.375889 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:25.375894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:25.375983 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:25.376108 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:25.376876 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.377888 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.380466 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.381801 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.381970 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:25.382087 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:25.382766 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 00:26:25.382936 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 00:26:25.384752 7816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:25.375337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:37757 (local address 127.2.120.3:40289)
0811 00:26:25.376846 (+ 1509us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:25.376891 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:25.377047 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:25.377423 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:25.377428 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:25.377454 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:25.378216 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:25.378377 (+ 161us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.379980 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.379984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:25.382087 (+ 2103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:25.382105 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.384008 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.384019 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:25.384154 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:25.384180 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 00:26:25.384268 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 00:26:25.384373 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1352,"mutex_wait_us":514,"thread_start_us":516,"threads_started":1}
I20260811 00:26:25.384539 7583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30c2130e9694ef9a95f0d9262015a07" candidate_uuid: "3f900da1efe04988aa68f68bf00a1de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1511f919ab134f45a2b48b54b9d85f36" is_pre_election: true
I20260811 00:26:25.384824 7803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:25.375356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:40289 (local address 127.2.120.1:37757)
0811 00:26:25.375569 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:25.375579 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:25.376998 (+ 1419us) server_negotiation.cc:407] Connection header received
0811 00:26:25.377217 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:25.377222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:25.377290 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:25.377403 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:25.378616 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.379768 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.382338 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.383741 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.383937 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:25.384088 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:25.384343 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 00:26:25.384538 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 00:26:25.385265 7583 raft_consensus.cc:2454] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f900da1efe04988aa68f68bf00a1de2 in term 0.
I20260811 00:26:25.386143 7655 leader_election.cc:302] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1511f919ab134f45a2b48b54b9d85f36, 3f900da1efe04988aa68f68bf00a1de2; no voters:
I20260811 00:26:25.386155 7425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30c2130e9694ef9a95f0d9262015a07" candidate_uuid: "3f900da1efe04988aa68f68bf00a1de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f64fe381c580438bae27a024f94afcc7" is_pre_election: true
I20260811 00:26:25.386623 7815 raft_consensus.cc:2781] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:26:25.386771 7815 raft_consensus.cc:482] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:26:25.386795 7425 raft_consensus.cc:2454] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f900da1efe04988aa68f68bf00a1de2 in term 0.
I20260811 00:26:25.386929 7815 raft_consensus.cc:3037] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:25.389991 7815 raft_consensus.cc:504] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1511f919ab134f45a2b48b54b9d85f36" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33333 } } peers { permanent_uuid: "3f900da1efe04988aa68f68bf00a1de2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36923 } } peers { permanent_uuid: "f64fe381c580438bae27a024f94afcc7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37757 } }
I20260811 00:26:25.390993 7815 leader_election.cc:288] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [CANDIDATE]: Term 1 election: Requested vote from peers 1511f919ab134f45a2b48b54b9d85f36 (127.2.120.2:33333), f64fe381c580438bae27a024f94afcc7 (127.2.120.1:37757)
I20260811 00:26:25.391562 7583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30c2130e9694ef9a95f0d9262015a07" candidate_uuid: "3f900da1efe04988aa68f68bf00a1de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1511f919ab134f45a2b48b54b9d85f36"
I20260811 00:26:25.391584 7425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d30c2130e9694ef9a95f0d9262015a07" candidate_uuid: "3f900da1efe04988aa68f68bf00a1de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f64fe381c580438bae27a024f94afcc7"
I20260811 00:26:25.391826 7583 raft_consensus.cc:3037] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:25.391839 7425 raft_consensus.cc:3037] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:25.394342 7425 raft_consensus.cc:2454] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f900da1efe04988aa68f68bf00a1de2 in term 1.
I20260811 00:26:25.394519 7583 raft_consensus.cc:2454] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f900da1efe04988aa68f68bf00a1de2 in term 1.
I20260811 00:26:25.395028 7656 leader_election.cc:302] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f900da1efe04988aa68f68bf00a1de2, f64fe381c580438bae27a024f94afcc7; no voters:
I20260811 00:26:25.395437 7815 raft_consensus.cc:2781] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:25.396046 7815 raft_consensus.cc:686] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [term 1 LEADER]: Becoming Leader. State: Replica: 3f900da1efe04988aa68f68bf00a1de2, State: Running, Role: LEADER
I20260811 00:26:25.396757 7815 consensus_queue.cc:237] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1511f919ab134f45a2b48b54b9d85f36" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33333 } } peers { permanent_uuid: "3f900da1efe04988aa68f68bf00a1de2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36923 } } peers { permanent_uuid: "f64fe381c580438bae27a024f94afcc7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37757 } }
I20260811 00:26:25.403029 7265 catalog_manager.cc:5495] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f900da1efe04988aa68f68bf00a1de2 (127.2.120.3). New cstate: current_term: 1 leader_uuid: "3f900da1efe04988aa68f68bf00a1de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1511f919ab134f45a2b48b54b9d85f36" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f900da1efe04988aa68f68bf00a1de2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f64fe381c580438bae27a024f94afcc7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37757 } health_report { overall_health: UNKNOWN } } }
I20260811 00:26:25.444317 7805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:25.436312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54236 (local address 127.2.120.3:36923)
0811 00:26:25.436717 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:25.436726 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:25.436947 (+ 221us) server_negotiation.cc:407] Connection header received
0811 00:26:25.437099 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:25.437103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:25.437208 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:25.437384 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:26:25.437940 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.438735 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.440923 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.441507 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.441638 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:25.442737 (+ 1099us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:26:25.443404 (+ 667us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:26:25.443598 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:25.443846 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 00:26:25.443944 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
W20260811 00:26:25.454252 7634 tablet.cc:2369] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:26:25.457510 7583 raft_consensus.cc:1261] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36 [term 1 FOLLOWER]: Refusing update from remote peer 3f900da1efe04988aa68f68bf00a1de2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:25.457508 7425 raft_consensus.cc:1261] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7 [term 1 FOLLOWER]: Refusing update from remote peer 3f900da1efe04988aa68f68bf00a1de2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:25.458786 7818 consensus_queue.cc:1035] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f64fe381c580438bae27a024f94afcc7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:25.459183 7815 consensus_queue.cc:1035] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1511f919ab134f45a2b48b54b9d85f36" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:25.475600 7822 mvcc.cc:205] Tried to move back new op lower bound from 7317127108416446464 to 7317127108189798400. Current Snapshot: MvccSnapshot[applied={T|T < 7317127108416446464}]
I20260811 00:26:25.476892 7825 mvcc.cc:205] Tried to move back new op lower bound from 7317127108416446464 to 7317127108189798400. Current Snapshot: MvccSnapshot[applied={T|T < 7317127108416446464}]
I20260811 00:26:25.492758 7266 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:25.493150 7266 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:25.498003 7266 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e103360b8d4f49dc90ce0a8d25899d1a: Sending DeleteTablet for 3 replicas of tablet d30c2130e9694ef9a95f0d9262015a07
I20260811 00:26:25.499578 7403 tablet_service.cc:1514] Processing DeleteTablet for tablet d30c2130e9694ef9a95f0d9262015a07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:25 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:57064
I20260811 00:26:25.499576 7557 tablet_service.cc:1514] Processing DeleteTablet for tablet d30c2130e9694ef9a95f0d9262015a07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:25 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:53402
I20260811 00:26:25.499835 7711 tablet_service.cc:1514] Processing DeleteTablet for tablet d30c2130e9694ef9a95f0d9262015a07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:25 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:54222
I20260811 00:26:25.500982 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 7335
I20260811 00:26:25.519665 7832 tablet_replica.cc:330] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7: stopping tablet replica
I20260811 00:26:25.520550 7832 raft_consensus.cc:2229] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:25.521415 7832 raft_consensus.cc:2258] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:25.527525 7832 ts_tablet_manager.cc:1905] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:25.528662 7834 tablet_replica.cc:330] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2: stopping tablet replica
I20260811 00:26:25.529608 7833 tablet_replica.cc:330] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36: stopping tablet replica
I20260811 00:26:25.529840 7834 raft_consensus.cc:2229] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:26:25.530251 7833 raft_consensus.cc:2229] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:25.530668 7834 raft_consensus.cc:2258] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:25.530974 7833 raft_consensus.cc:2258] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:25.533798 7833 ts_tablet_manager.cc:1905] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 00:26:25 dist-test-slave-fd7l krb5kdc[7218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:26:25.538512 7832 ts_tablet_manager.cc:1918] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:26:25.538978 7834 ts_tablet_manager.cc:1905] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:25.539108 7832 log.cc:1196] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/d30c2130e9694ef9a95f0d9262015a07
I20260811 00:26:25.540146 7832 ts_tablet_manager.cc:1939] T d30c2130e9694ef9a95f0d9262015a07 P f64fe381c580438bae27a024f94afcc7: Deleting consensus metadata
I20260811 00:26:25.542603 7252 catalog_manager.cc:4841] TS f64fe381c580438bae27a024f94afcc7 (127.2.120.1:37757): tablet d30c2130e9694ef9a95f0d9262015a07 (table test-table [id=6aa1f1c554c54524bb2075b1770f3a58]) successfully deleted
I20260811 00:26:25.547554 7833 ts_tablet_manager.cc:1918] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:26:25.547596 7834 ts_tablet_manager.cc:1918] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:26:25.547782 7833 log.cc:1196] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/d30c2130e9694ef9a95f0d9262015a07
I20260811 00:26:25.547806 7834 log.cc:1196] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/d30c2130e9694ef9a95f0d9262015a07
I20260811 00:26:25.548442 7833 ts_tablet_manager.cc:1939] T d30c2130e9694ef9a95f0d9262015a07 P 1511f919ab134f45a2b48b54b9d85f36: Deleting consensus metadata
I20260811 00:26:25.548470 7834 ts_tablet_manager.cc:1939] T d30c2130e9694ef9a95f0d9262015a07 P 3f900da1efe04988aa68f68bf00a1de2: Deleting consensus metadata
I20260811 00:26:25.548764 7803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:25.503661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57076 (local address 127.2.120.1:37757)
0811 00:26:25.504162 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:25.504173 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:25.514907 (+ 10734us) server_negotiation.cc:407] Connection header received
0811 00:26:25.515670 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:25.515681 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:25.515836 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:25.515971 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:25.518830 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.520820 (+ 1990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.524612 (+ 3792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.525813 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.526015 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:25.538083 (+ 12068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:25.538104 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:25.538126 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:25.538183 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:25.543727 (+ 5544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:25.544881 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:25.544893 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:25.544897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:25.545103 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:25.545624 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:25.545629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:25.545633 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:25.546153 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:25.546491 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:25.547570 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0811 00:26:25.548139 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
I20260811 00:26:25.550263 7469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:25.550634 7251 catalog_manager.cc:4841] TS 1511f919ab134f45a2b48b54b9d85f36 (127.2.120.2:33333): tablet d30c2130e9694ef9a95f0d9262015a07 (table test-table [id=6aa1f1c554c54524bb2075b1770f3a58]) successfully deleted
I20260811 00:26:25.551699 7251 catalog_manager.cc:4841] TS 3f900da1efe04988aa68f68bf00a1de2 (127.2.120.3:36923): tablet d30c2130e9694ef9a95f0d9262015a07 (table test-table [id=6aa1f1c554c54524bb2075b1770f3a58]) successfully deleted
I20260811 00:26:25.649437 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 7335
I20260811 00:26:25.673210 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 7482
Aug 11 00:26:25 dist-test-slave-fd7l krb5kdc[7218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.2@KRBTEST.COM
I20260811 00:26:25.694080 7800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:25.674522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53404 (local address 127.2.120.2:33333)
0811 00:26:25.674742 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:25.674747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:25.674847 (+ 100us) server_negotiation.cc:407] Connection header received
0811 00:26:25.675296 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:25.675301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:25.675378 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:25.675494 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:25.677393 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.678231 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.680401 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.681041 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.681228 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:25.687079 (+ 5851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:25.687099 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:25.687120 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:25.687180 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:25.690551 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:25.691588 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:25.691597 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:25.691599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:25.691756 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:25.692179 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:25.692184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:25.692188 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:25.692708 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:25.693054 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:25.693431 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 00:26:25.693715 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 00:26:25.694784 7628 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:25.789901 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 7482
I20260811 00:26:25.821868 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 7639
Aug 11 00:26:25 dist-test-slave-fd7l krb5kdc[7218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.3@KRBTEST.COM
I20260811 00:26:25.846647 7805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:25.823064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54242 (local address 127.2.120.3:36923)
0811 00:26:25.823498 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:25.823503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:25.823531 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:26:25.823672 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:25.823676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:25.823898 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:25.824025 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:25.826017 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.827184 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.829616 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:25.830276 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:25.830451 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:25.837232 (+ 6781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:25.837256 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:25.837285 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:25.837356 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:25.842113 (+ 4757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:25.843387 (+ 1274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:25.843398 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:25.843402 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:25.843618 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:25.844619 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:25.844624 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:25.844627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:25.845135 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:25.845598 (+ 463us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:25.845911 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 00:26:25.846194 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266}
I20260811 00:26:25.847508 7778 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:25.980863 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 7639
I20260811 00:26:26.005306 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 7233
I20260811 00:26:26.006705 7301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:26.119472 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 7233
2026-08-11T00:26:26Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (5123 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T00:26:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:26:26Z Disabled control of system clock
I20260811 00:26:26.206489 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:43431
--webserver_interface=127.2.120.62
--webserver_port=0
--builtin_ntp_servers=127.2.120.20:37127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:43431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 00:26:26.583678 7843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:26.584139 7843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:26.584280 7843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:26.591957 7843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:26:26.592085 7843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:26.592123 7843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:26.592211 7843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:26:26.592278 7843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:26:26.602212 7843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:43431
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:43431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:26.603760 7843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:26.605911 7843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:26.620468 7853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:26.620445 7851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:26.620908 7849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:26.622391 7843 server_base.cc:1034] running on GCE node
I20260811 00:26:26.623476 7843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:26.625484 7843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:26.626797 7843 hybrid_clock.cc:648] HybridClock initialized: now 1786407986626741 us; error 42 us; skew 500 ppm
I20260811 00:26:26.627240 7843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:26:26.630466 7843 webserver.cc:460] Webserver started at http://127.2.120.62:39297/ using document root <none> and password file <none>
I20260811 00:26:26.631371 7843 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:26.631546 7843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:26.631930 7843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:26.635535 7843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "21a2be276fb24d72a683fe1105116fae"
format_stamp: "Formatted at 2026-08-11 00:26:26 on dist-test-slave-fd7l"
server_key: "b40f14e5cdcf564f10a32c2c0e611563"
server_key_iv: "bdd3cc8728417a017e59a3431cc1ce70"
server_key_version: "encryptionkey@0"
I20260811 00:26:26.637125 7843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "21a2be276fb24d72a683fe1105116fae"
format_stamp: "Formatted at 2026-08-11 00:26:26 on dist-test-slave-fd7l"
server_key: "b40f14e5cdcf564f10a32c2c0e611563"
server_key_iv: "bdd3cc8728417a017e59a3431cc1ce70"
server_key_version: "encryptionkey@0"
I20260811 00:26:26.645581 7843 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.005s
I20260811 00:26:26.650799 7860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:26.653323 7843 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 00:26:26.653712 7843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "21a2be276fb24d72a683fe1105116fae"
format_stamp: "Formatted at 2026-08-11 00:26:26 on dist-test-slave-fd7l"
server_key: "b40f14e5cdcf564f10a32c2c0e611563"
server_key_iv: "bdd3cc8728417a017e59a3431cc1ce70"
server_key_version: "encryptionkey@0"
I20260811 00:26:26.654073 7843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:26.662786 7843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:26.663975 7843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:26.664304 7843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:26.690158 7843 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:43431
I20260811 00:26:26.690174 7919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:43431 every 8 connection(s)
I20260811 00:26:26.691907 7843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:26:26.694247 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 7843
I20260811 00:26:26.694484 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:26:26.695421 2528 external_mini_cluster.cc:1428] Setting key 9e253ecfe7e57c653a890606244b3f49
I20260811 00:26:26.697858 7920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:26.708125 7923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:26.698139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40142 (local address 127.2.120.62:43431)
0811 00:26:26.699817 (+ 1678us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:26.699869 (+ 52us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:26.699955 (+ 86us) server_negotiation.cc:407] Connection header received
0811 00:26:26.702472 (+ 2517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:26.702512 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:26.703158 (+ 646us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:26.703743 (+ 585us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:26.705231 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:26.705238 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:26.705255 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:26.705273 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:26.705377 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:26.705398 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:26.705883 (+ 485us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:26.706178 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 00:26:26.706692 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":844,"thread_start_us":533,"threads_started":1}
I20260811 00:26:26.716020 7920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae: Bootstrap starting.
I20260811 00:26:26.720440 7920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:26.722419 7920 log.cc:824] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:26.726722 7920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae: No bootstrap required, opened a new log
I20260811 00:26:26.735194 7920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a2be276fb24d72a683fe1105116fae" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43431 } }
I20260811 00:26:26.735651 7920 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:26.735770 7920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a2be276fb24d72a683fe1105116fae, State: Initialized, Role: FOLLOWER
I20260811 00:26:26.736636 7920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a2be276fb24d72a683fe1105116fae" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43431 } }
I20260811 00:26:26.736929 7920 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:26:26.737092 7920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:26:26.737290 7920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:26.740579 7920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a2be276fb24d72a683fe1105116fae" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43431 } }
I20260811 00:26:26.741358 7920 leader_election.cc:302] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21a2be276fb24d72a683fe1105116fae; no voters:
I20260811 00:26:26.742072 7920 leader_election.cc:288] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:26:26.742255 7925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:26.743135 7925 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae [term 1 LEADER]: Becoming Leader. State: Replica: 21a2be276fb24d72a683fe1105116fae, State: Running, Role: LEADER
I20260811 00:26:26.743816 7925 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a2be276fb24d72a683fe1105116fae" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43431 } }
I20260811 00:26:26.744834 7920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:26:26.747537 7927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21a2be276fb24d72a683fe1105116fae. Latest consensus state: current_term: 1 leader_uuid: "21a2be276fb24d72a683fe1105116fae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a2be276fb24d72a683fe1105116fae" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43431 } } }
I20260811 00:26:26.748102 7927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:26.748698 7926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21a2be276fb24d72a683fe1105116fae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a2be276fb24d72a683fe1105116fae" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 43431 } } }
I20260811 00:26:26.749262 7926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:26.749965 7934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:26:26.757237 7934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:26:26.769721 7934 catalog_manager.cc:1347] Generated new cluster ID: b1d1e47f8d6d4c1086e2620ccae4e0ef
I20260811 00:26:26.770027 7934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:26:26.782028 7934 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:26:26.782586 7934 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:26:26.801083 7934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae: Generated new TSK 0
I20260811 00:26:26.802446 7934 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:26:26.821588 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43431
--builtin_ntp_servers=127.2.120.20:37127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 00:26:27.248776 7945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:27.249246 7945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:27.249460 7945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:27.257424 7945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:27.257639 7945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:27.257838 7945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:26:27.268668 7945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:27.270196 7945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:27.272330 7945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:27.288342 7951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:27.288342 7952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:27.288348 7954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:27.289407 7945 server_base.cc:1034] running on GCE node
I20260811 00:26:27.290126 7945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:27.291285 7945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:27.292629 7945 hybrid_clock.cc:648] HybridClock initialized: now 1786407987292560 us; error 52 us; skew 500 ppm
I20260811 00:26:27.293004 7945 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:26:27.295825 7945 webserver.cc:460] Webserver started at http://127.2.120.1:33647/ using document root <none> and password file <none>
I20260811 00:26:27.296777 7945 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:27.296933 7945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:27.297305 7945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:27.300998 7945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "a199d558849b4074a1513877d7f71043"
format_stamp: "Formatted at 2026-08-11 00:26:27 on dist-test-slave-fd7l"
server_key: "c75078eadb48a504053be7bbe6c5641c"
server_key_iv: "2fde00375260d958db8092f79ac0dc3b"
server_key_version: "encryptionkey@0"
I20260811 00:26:27.302165 7945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "a199d558849b4074a1513877d7f71043"
format_stamp: "Formatted at 2026-08-11 00:26:27 on dist-test-slave-fd7l"
server_key: "c75078eadb48a504053be7bbe6c5641c"
server_key_iv: "2fde00375260d958db8092f79ac0dc3b"
server_key_version: "encryptionkey@0"
I20260811 00:26:27.308320 7945 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 00:26:27.312987 7963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:27.314682 7945 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:26:27.314929 7945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "a199d558849b4074a1513877d7f71043"
format_stamp: "Formatted at 2026-08-11 00:26:27 on dist-test-slave-fd7l"
server_key: "c75078eadb48a504053be7bbe6c5641c"
server_key_iv: "2fde00375260d958db8092f79ac0dc3b"
server_key_version: "encryptionkey@0"
I20260811 00:26:27.315231 7945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:27.322301 7945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:27.323333 7945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:27.323619 7945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:27.324730 7945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:27.326489 7945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:27.326597 7945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:27.326681 7945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:27.326741 7945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:27.366786 7945 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:44811
I20260811 00:26:27.366843 8089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:44811 every 8 connection(s)
I20260811 00:26:27.368301 7945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:26:27.373996 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 7945
I20260811 00:26:27.374228 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:26:27.375005 2528 external_mini_cluster.cc:1428] Setting key ed7a52c0f1628f2e2f11cd91ccef4e36
I20260811 00:26:27.378060 8092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:27.371242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:33273 (local address 127.2.120.62:43431)
0811 00:26:27.371771 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:27.371776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:27.373208 (+ 1432us) server_negotiation.cc:407] Connection header received
0811 00:26:27.375313 (+ 2105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:27.375319 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:27.375411 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:27.375541 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:27.377109 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:27.377125 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:27.377130 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:27.377157 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:27.377232 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:27.377246 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:27.377391 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:27.377606 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 00:26:27.377683 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":286,"threads_started":1}
I20260811 00:26:27.379235 8093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:27.371877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:43431 (local address 127.2.120.1:33273)
0811 00:26:27.373011 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:27.373068 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:27.374943 (+ 1875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:27.375832 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:27.375841 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:27.375951 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:27.376762 (+ 811us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:26:27.376765 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:27.376808 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:26:27.376910 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:27.377361 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:27.377373 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:27.377382 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 00:26:27.377720 (+ 338us) client_negotiation.cc:240] Negotiation successful
0811 00:26:27.378041 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":337,"threads_started":1}
I20260811 00:26:27.379220 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43431
--builtin_ntp_servers=127.2.120.20:37127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 00:26:27.382448 8090 heartbeater.cc:347] Connected to a master server at 127.2.120.62:43431
I20260811 00:26:27.382936 8090 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:27.383848 8090 heartbeater.cc:511] Master 127.2.120.62:43431 requested a full tablet report, sending...
I20260811 00:26:27.386967 7880 ts_manager.cc:194] Registered new tserver with Master: a199d558849b4074a1513877d7f71043 (127.2.120.1:44811)
W20260811 00:26:27.780311 8094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:27.780752 8094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:27.780916 8094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:27.789041 8094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:27.789181 8094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:27.789371 8094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:26:27.799276 8094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:27.800869 8094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:27.802892 8094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:27.819877 8102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:27.819713 8104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:27.819739 8101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:27.820827 8094 server_base.cc:1034] running on GCE node
I20260811 00:26:27.821621 8094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:27.823060 8094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:27.824463 8094 hybrid_clock.cc:648] HybridClock initialized: now 1786407987824368 us; error 78 us; skew 500 ppm
I20260811 00:26:27.825059 8094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:26:27.827831 8094 webserver.cc:460] Webserver started at http://127.2.120.2:36181/ using document root <none> and password file <none>
I20260811 00:26:27.828836 8094 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:27.828994 8094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:27.829360 8094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:27.833030 8094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "8de08b4f4047467186e7181ef1252d0f"
format_stamp: "Formatted at 2026-08-11 00:26:27 on dist-test-slave-fd7l"
server_key: "4dc2e1612e44b4ce8eac8da55fd8f18e"
server_key_iv: "b82db61b590553fd45fb5ae43cdca32e"
server_key_version: "encryptionkey@0"
I20260811 00:26:27.833982 8094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "8de08b4f4047467186e7181ef1252d0f"
format_stamp: "Formatted at 2026-08-11 00:26:27 on dist-test-slave-fd7l"
server_key: "4dc2e1612e44b4ce8eac8da55fd8f18e"
server_key_iv: "b82db61b590553fd45fb5ae43cdca32e"
server_key_version: "encryptionkey@0"
I20260811 00:26:27.840210 8094 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 00:26:27.845378 8111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:27.848109 8094 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.002s
I20260811 00:26:27.848618 8094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "8de08b4f4047467186e7181ef1252d0f"
format_stamp: "Formatted at 2026-08-11 00:26:27 on dist-test-slave-fd7l"
server_key: "4dc2e1612e44b4ce8eac8da55fd8f18e"
server_key_iv: "b82db61b590553fd45fb5ae43cdca32e"
server_key_version: "encryptionkey@0"
I20260811 00:26:27.849020 8094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:27.856967 8094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:27.858000 8094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:27.858340 8094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:27.859742 8094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:27.861598 8094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:27.861729 8094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:27.861886 8094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:27.861984 8094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:27.907879 8094 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:35461
I20260811 00:26:27.907917 8233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:35461 every 8 connection(s)
I20260811 00:26:27.910018 8094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:26:27.910568 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 8094
I20260811 00:26:27.910804 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:26:27.911470 2528 external_mini_cluster.cc:1428] Setting key 67e8cb4b046e9ee4a486a78f75f2dba4
I20260811 00:26:27.916580 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43431
--builtin_ntp_servers=127.2.120.20:37127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 00:26:27.921450 8236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:27.914071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:34771 (local address 127.2.120.62:43431)
0811 00:26:27.914710 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:27.914717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:27.916084 (+ 1367us) server_negotiation.cc:407] Connection header received
0811 00:26:27.918195 (+ 2111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:27.918201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:27.918305 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:27.918438 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:27.920481 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:27.920489 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:27.920493 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:27.920545 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:27.920612 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:27.920629 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:27.920842 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:27.921038 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 00:26:27.921117 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":336,"threads_started":1}
I20260811 00:26:27.922608 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:27.914781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:43431 (local address 127.2.120.2:34771)
0811 00:26:27.915903 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:27.915947 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:27.917816 (+ 1869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:27.918751 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:27.918762 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:27.918872 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:27.920057 (+ 1185us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:26:27.920061 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:27.920135 (+ 74us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:26:27.920254 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:27.920766 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:27.920781 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:27.920790 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 00:26:27.921152 (+ 362us) client_negotiation.cc:240] Negotiation successful
0811 00:26:27.921490 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":394,"threads_started":1}
I20260811 00:26:27.924620 8234 heartbeater.cc:347] Connected to a master server at 127.2.120.62:43431
I20260811 00:26:27.925019 8234 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:27.925752 8234 heartbeater.cc:511] Master 127.2.120.62:43431 requested a full tablet report, sending...
I20260811 00:26:27.927528 7880 ts_manager.cc:194] Registered new tserver with Master: 8de08b4f4047467186e7181ef1252d0f (127.2.120.2:35461)
W20260811 00:26:28.302635 8238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:28.303063 8238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:28.303246 8238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:28.310925 8238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:28.311107 8238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:28.311295 8238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:26:28.321326 8238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:43431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:28.322842 8238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:28.324882 8238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:28.339210 8244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:28.339273 8245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:28.339185 8247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:28.340320 8238 server_base.cc:1034] running on GCE node
I20260811 00:26:28.341115 8238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:28.342356 8238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:28.343681 8238 hybrid_clock.cc:648] HybridClock initialized: now 1786407988343591 us; error 69 us; skew 500 ppm
I20260811 00:26:28.344182 8238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:26:28.347757 8238 webserver.cc:460] Webserver started at http://127.2.120.3:35695/ using document root <none> and password file <none>
I20260811 00:26:28.348973 8238 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:28.349097 8238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:28.349399 8238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:28.352660 8238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "250108779aeb4d4fadcb6aae04f5ca73"
format_stamp: "Formatted at 2026-08-11 00:26:28 on dist-test-slave-fd7l"
server_key: "a8956107dc76375caa050fd4f982958f"
server_key_iv: "d89590261b79bd7483823b03113e5819"
server_key_version: "encryptionkey@0"
I20260811 00:26:28.353497 8238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "250108779aeb4d4fadcb6aae04f5ca73"
format_stamp: "Formatted at 2026-08-11 00:26:28 on dist-test-slave-fd7l"
server_key: "a8956107dc76375caa050fd4f982958f"
server_key_iv: "d89590261b79bd7483823b03113e5819"
server_key_version: "encryptionkey@0"
I20260811 00:26:28.359660 8238 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 00:26:28.363922 8253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:28.365824 8238 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 00:26:28.366043 8238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "250108779aeb4d4fadcb6aae04f5ca73"
format_stamp: "Formatted at 2026-08-11 00:26:28 on dist-test-slave-fd7l"
server_key: "a8956107dc76375caa050fd4f982958f"
server_key_iv: "d89590261b79bd7483823b03113e5819"
server_key_version: "encryptionkey@0"
I20260811 00:26:28.366385 8238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:28.372843 8238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:28.373767 8238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:28.374081 8238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:28.375274 8238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:28.376983 8238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:28.377084 8238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:28.377210 8238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:28.377308 8238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:28.390648 8090 heartbeater.cc:503] Master 127.2.120.62:43431 was elected leader, sending a full tablet report...
I20260811 00:26:28.414901 8238 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:39345
I20260811 00:26:28.414958 8380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:39345 every 8 connection(s)
I20260811 00:26:28.416323 8238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:26:28.425374 8236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:28.419055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:53423 (local address 127.2.120.62:43431)
0811 00:26:28.419231 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:28.419235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:28.420977 (+ 1742us) server_negotiation.cc:407] Connection header received
0811 00:26:28.422806 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:28.422810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:28.422867 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:28.422970 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:28.424642 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:28.424646 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:28.424650 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:28.424668 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:28.424702 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:28.424715 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:28.424836 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:28.425079 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 00:26:28.425138 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 00:26:28.426178 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 8238
I20260811 00:26:28.426354 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:26:28.426620 8384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:28.419595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:43431 (local address 127.2.120.3:53423)
0811 00:26:28.420725 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:28.420803 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:28.422491 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:28.423394 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:28.423408 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:28.423519 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:28.424221 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:26:28.424224 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:28.424271 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:26:28.424384 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:28.424819 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:28.424828 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:28.424835 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:26:28.425085 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 00:26:28.425314 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":288,"threads_started":1}
I20260811 00:26:28.426901 2528 external_mini_cluster.cc:1428] Setting key 82bf4b2df65c1d76802f25fed3a8bfa5
I20260811 00:26:28.428359 8382 heartbeater.cc:347] Connected to a master server at 127.2.120.62:43431
I20260811 00:26:28.428771 8382 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:28.429543 8382 heartbeater.cc:511] Master 127.2.120.62:43431 requested a full tablet report, sending...
I20260811 00:26:28.431173 7880 ts_manager.cc:194] Registered new tserver with Master: 250108779aeb4d4fadcb6aae04f5ca73 (127.2.120.3:39345)
I20260811 00:26:28.444172 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:26:28.452989 8236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:28.450018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40146 (local address 127.2.120.62:43431)
0811 00:26:28.450197 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:28.450201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:28.450859 (+ 658us) server_negotiation.cc:407] Connection header received
0811 00:26:28.450972 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:28.450976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:28.451034 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:28.451130 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:28.452210 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:28.452215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:28.452219 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:28.452235 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:28.452269 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:28.452281 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:28.452389 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:28.452683 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 00:26:28.452737 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 00:26:28.457713 7880 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:26:28.465121 7880 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:26:28.486493 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:28.481148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:44811 (local address 127.0.0.1:33034)
0811 00:26:28.481888 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:28.481925 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:28.482081 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:28.483171 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:28.483179 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:28.483281 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:28.484317 (+ 1036us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:26:28.484321 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:28.484352 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:26:28.484449 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:28.485663 (+ 1214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:28.485682 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:28.485691 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 00:26:28.485857 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 00:26:28.485970 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":752,"threads_started":1}
I20260811 00:26:28.487686 8400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:28.481704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33034 (local address 127.2.120.1:44811)
0811 00:26:28.482641 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:28.482652 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:28.482681 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:26:28.482747 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:28.482753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:28.482938 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:28.483828 (+ 890us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:28.485383 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:28.485392 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:28.485408 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:28.485427 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:28.485493 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:28.485519 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:28.487106 (+ 1587us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:28.487184 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 00:26:28.487310 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":620,"threads_started":1}
I20260811 00:26:28.489018 8397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:28.480428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41886 (local address 127.2.120.3:39345)
0811 00:26:28.481142 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:28.481151 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:28.482042 (+ 891us) server_negotiation.cc:407] Connection header received
0811 00:26:28.482292 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:28.482303 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:28.482595 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:28.482731 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:28.486486 (+ 3755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:28.486498 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:28.486513 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:28.486537 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:28.486722 (+ 185us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:28.486747 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:28.488231 (+ 1484us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:28.488485 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 00:26:28.488634 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":557,"threads_started":1}
I20260811 00:26:28.489100 8396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:28.480234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:39345 (local address 127.0.0.1:41886)
0811 00:26:28.481878 (+ 1644us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:28.481925 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:28.482113 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:28.485360 (+ 3247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:28.485364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:28.485390 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:28.486197 (+ 807us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:26:28.486201 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:28.486218 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:26:28.486275 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:28.488322 (+ 2047us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:28.488325 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:28.488328 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 00:26:28.488487 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 00:26:28.488652 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1330,"thread_start_us":1116,"threads_started":1}
I20260811 00:26:28.493080 8016 tablet_service.cc:1467] Processing CreateTablet for tablet d14c8023df7e49bb9338a47791ef1942 (DEFAULT_TABLE table=test-table [id=370f18175dbc4559b022a1368e12a662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:28.494560 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:28.482410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:35461 (local address 127.0.0.1:52698)
0811 00:26:28.486760 (+ 4350us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:28.486781 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:28.486860 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:28.492144 (+ 5284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:28.492150 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:28.492201 (+ 51us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:28.492943 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:26:28.492947 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:28.492968 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:26:28.493025 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:28.494028 (+ 1003us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:28.494036 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:28.494043 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:26:28.494185 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 00:26:28.494252 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4256,"thread_start_us":8340,"threads_started":1}
I20260811 00:26:28.494639 8402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:28.483052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52698 (local address 127.2.120.2:35461)
0811 00:26:28.491521 (+ 8469us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:28.491531 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:28.491565 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:26:28.491643 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:28.491651 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:28.491900 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:28.492829 (+ 929us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:28.493759 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:28.493769 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:28.493783 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:28.493800 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:28.493854 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:28.493879 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:28.494052 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:28.494222 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 00:26:28.494334 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8283,"thread_start_us":8146,"threads_started":1}
I20260811 00:26:28.494823 8016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14c8023df7e49bb9338a47791ef1942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:28.495057 8309 tablet_service.cc:1467] Processing CreateTablet for tablet d14c8023df7e49bb9338a47791ef1942 (DEFAULT_TABLE table=test-table [id=370f18175dbc4559b022a1368e12a662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:28.497458 8309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14c8023df7e49bb9338a47791ef1942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:28.498853 8160 tablet_service.cc:1467] Processing CreateTablet for tablet d14c8023df7e49bb9338a47791ef1942 (DEFAULT_TABLE table=test-table [id=370f18175dbc4559b022a1368e12a662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:28.500602 8160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14c8023df7e49bb9338a47791ef1942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:28.508380 8403 tablet_bootstrap.cc:492] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043: Bootstrap starting.
I20260811 00:26:28.512492 8403 tablet_bootstrap.cc:654] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:28.513847 8403 log.cc:824] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:28.516577 8405 tablet_bootstrap.cc:492] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73: Bootstrap starting.
I20260811 00:26:28.517412 8403 tablet_bootstrap.cc:492] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043: No bootstrap required, opened a new log
I20260811 00:26:28.517737 8403 ts_tablet_manager.cc:1397] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 00:26:28.519922 8406 tablet_bootstrap.cc:492] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f: Bootstrap starting.
I20260811 00:26:28.522218 8405 tablet_bootstrap.cc:654] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:28.524187 8405 log.cc:824] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:28.525136 8406 tablet_bootstrap.cc:654] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:28.526830 8406 log.cc:824] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:28.527709 8403 raft_consensus.cc:348] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250108779aeb4d4fadcb6aae04f5ca73" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 39345 } } peers { permanent_uuid: "8de08b4f4047467186e7181ef1252d0f" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 35461 } } peers { permanent_uuid: "a199d558849b4074a1513877d7f71043" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44811 } }
I20260811 00:26:28.528302 8403 raft_consensus.cc:374] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:28.528438 8403 raft_consensus.cc:729] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a199d558849b4074a1513877d7f71043, State: Initialized, Role: FOLLOWER
I20260811 00:26:28.529289 8405 tablet_bootstrap.cc:492] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73: No bootstrap required, opened a new log
I20260811 00:26:28.529361 8403 consensus_queue.cc:260] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250108779aeb4d4fadcb6aae04f5ca73" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 39345 } } peers { permanent_uuid: "8de08b4f4047467186e7181ef1252d0f" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 35461 } } peers { permanent_uuid: "a199d558849b4074a1513877d7f71043" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44811 } }
I20260811 00:26:28.529700 8405 ts_tablet_manager.cc:1397] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.008s
I20260811 00:26:28.531093 8406 tablet_bootstrap.cc:492] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f: No bootstrap required, opened a new log
I20260811 00:26:28.531915 8406 ts_tablet_manager.cc:1397] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260811 00:26:28.533272 8403 ts_tablet_manager.cc:1428] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260811 00:26:28.539985 8405 raft_consensus.cc:348] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250108779aeb4d4fadcb6aae04f5ca73" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 39345 } } peers { permanent_uuid: "8de08b4f4047467186e7181ef1252d0f" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 35461 } } peers { permanent_uuid: "a199d558849b4074a1513877d7f71043" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44811 } }
I20260811 00:26:28.540441 8405 raft_consensus.cc:374] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:28.540621 8405 raft_consensus.cc:729] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 250108779aeb4d4fadcb6aae04f5ca73, State: Initialized, Role: FOLLOWER
I20260811 00:26:28.541252 8405 consensus_queue.cc:260] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250108779aeb4d4fadcb6aae04f5ca73" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 39345 } } peers { permanent_uuid: "8de08b4f4047467186e7181ef1252d0f" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 35461 } } peers { permanent_uuid: "a199d558849b4074a1513877d7f71043" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44811 } }
I20260811 00:26:28.542500 8382 heartbeater.cc:503] Master 127.2.120.62:43431 was elected leader, sending a full tablet report...
I20260811 00:26:28.543222 8405 ts_tablet_manager.cc:1428] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260811 00:26:28.543720 8406 raft_consensus.cc:348] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250108779aeb4d4fadcb6aae04f5ca73" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 39345 } } peers { permanent_uuid: "8de08b4f4047467186e7181ef1252d0f" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 35461 } } peers { permanent_uuid: "a199d558849b4074a1513877d7f71043" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44811 } }
I20260811 00:26:28.544190 8406 raft_consensus.cc:374] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:28.544305 8406 raft_consensus.cc:729] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8de08b4f4047467186e7181ef1252d0f, State: Initialized, Role: FOLLOWER
I20260811 00:26:28.545120 8406 consensus_queue.cc:260] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250108779aeb4d4fadcb6aae04f5ca73" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 39345 } } peers { permanent_uuid: "8de08b4f4047467186e7181ef1252d0f" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 35461 } } peers { permanent_uuid: "a199d558849b4074a1513877d7f71043" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44811 } }
I20260811 00:26:28.546883 8234 heartbeater.cc:503] Master 127.2.120.62:43431 was elected leader, sending a full tablet report...
I20260811 00:26:28.548985 8406 ts_tablet_manager.cc:1428] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f: Time spent starting tablet: real 0.017s user 0.014s sys 0.000s
W20260811 00:26:28.621315 8091 tablet.cc:2369] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:26:28.655001 8409 raft_consensus.cc:482] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:26:28.655365 8409 raft_consensus.cc:504] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250108779aeb4d4fadcb6aae04f5ca73" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 39345 } } peers { permanent_uuid: "8de08b4f4047467186e7181ef1252d0f" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 35461 } } peers { permanent_uuid: "a199d558849b4074a1513877d7f71043" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44811 } }
I20260811 00:26:28.657440 8409 leader_election.cc:288] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 250108779aeb4d4fadcb6aae04f5ca73 (127.2.120.3:39345), 8de08b4f4047467186e7181ef1252d0f (127.2.120.2:35461)
I20260811 00:26:28.661535 8397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:28.658088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:44447 (local address 127.2.120.3:39345)
0811 00:26:28.658285 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:28.658290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:28.658685 (+ 395us) server_negotiation.cc:407] Connection header received
0811 00:26:28.658861 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:28.658871 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:28.658953 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:28.659089 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:28.660856 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:28.660861 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:28.660865 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:28.660882 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:28.660914 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:28.660926 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:28.661073 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:28.661162 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 00:26:28.661234 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:26:28.661721 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:28.657733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:39345 (local address 127.2.120.1:44447)
0811 00:26:28.658516 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:28.658540 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:28.658681 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:28.659317 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:28.659324 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:28.659354 (+ 30us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:28.660595 (+ 1241us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:26:28.660599 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:28.660627 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:26:28.660675 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:28.661040 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:28.661047 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:28.661054 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:26:28.661203 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 00:26:28.661257 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":403,"threads_started":1}
I20260811 00:26:28.662182 8402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:28.659032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:42081 (local address 127.2.120.2:35461)
0811 00:26:28.659264 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:28.659270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:28.659521 (+ 251us) server_negotiation.cc:407] Connection header received
0811 00:26:28.659677 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:28.659684 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:28.659755 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:28.659883 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:28.661444 (+ 1561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:28.661450 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:28.661455 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:28.661474 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:28.661511 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:28.661523 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:28.661659 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:28.661841 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 00:26:28.661916 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:26:28.662513 8413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:28.658669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:35461 (local address 127.2.120.1:42081)
0811 00:26:28.659431 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:28.659447 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:28.659550 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:28.660200 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:28.660208 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:28.660252 (+ 44us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:28.661215 (+ 963us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:26:28.661218 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:28.661232 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:26:28.661270 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:28.661635 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:28.661639 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:28.661645 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 00:26:28.661880 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 00:26:28.661945 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":404,"threads_started":1}
I20260811 00:26:28.663167 8331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14c8023df7e49bb9338a47791ef1942" candidate_uuid: "a199d558849b4074a1513877d7f71043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "250108779aeb4d4fadcb6aae04f5ca73" is_pre_election: true
W20260811 00:26:28.663439 8235 tablet.cc:2369] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:26:28.663794 8331 raft_consensus.cc:2454] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a199d558849b4074a1513877d7f71043 in term 0.
I20260811 00:26:28.663741 8185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14c8023df7e49bb9338a47791ef1942" candidate_uuid: "a199d558849b4074a1513877d7f71043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8de08b4f4047467186e7181ef1252d0f" is_pre_election: true
I20260811 00:26:28.664316 8185 raft_consensus.cc:2454] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a199d558849b4074a1513877d7f71043 in term 0.
I20260811 00:26:28.664651 7966 leader_election.cc:302] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 250108779aeb4d4fadcb6aae04f5ca73, a199d558849b4074a1513877d7f71043; no voters:
I20260811 00:26:28.665241 8409 raft_consensus.cc:2781] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:26:28.665439 8409 raft_consensus.cc:482] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:26:28.665575 8409 raft_consensus.cc:3037] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [term 0 FOLLOWER]: Advancing to term 1
W20260811 00:26:28.668195 8383 tablet.cc:2369] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:26:28.668257 8409 raft_consensus.cc:504] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250108779aeb4d4fadcb6aae04f5ca73" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 39345 } } peers { permanent_uuid: "8de08b4f4047467186e7181ef1252d0f" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 35461 } } peers { permanent_uuid: "a199d558849b4074a1513877d7f71043" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44811 } }
I20260811 00:26:28.669407 8409 leader_election.cc:288] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [CANDIDATE]: Term 1 election: Requested vote from peers 250108779aeb4d4fadcb6aae04f5ca73 (127.2.120.3:39345), 8de08b4f4047467186e7181ef1252d0f (127.2.120.2:35461)
I20260811 00:26:28.670311 8185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14c8023df7e49bb9338a47791ef1942" candidate_uuid: "a199d558849b4074a1513877d7f71043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8de08b4f4047467186e7181ef1252d0f"
I20260811 00:26:28.670307 8331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14c8023df7e49bb9338a47791ef1942" candidate_uuid: "a199d558849b4074a1513877d7f71043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "250108779aeb4d4fadcb6aae04f5ca73"
I20260811 00:26:28.670568 8331 raft_consensus.cc:3037] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:28.670562 8185 raft_consensus.cc:3037] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:28.674343 8185 raft_consensus.cc:2454] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a199d558849b4074a1513877d7f71043 in term 1.
I20260811 00:26:28.674343 8331 raft_consensus.cc:2454] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a199d558849b4074a1513877d7f71043 in term 1.
I20260811 00:26:28.675035 7966 leader_election.cc:302] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8de08b4f4047467186e7181ef1252d0f, a199d558849b4074a1513877d7f71043; no voters:
I20260811 00:26:28.675413 8409 raft_consensus.cc:2781] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:28.676079 8409 raft_consensus.cc:686] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [term 1 LEADER]: Becoming Leader. State: Replica: a199d558849b4074a1513877d7f71043, State: Running, Role: LEADER
I20260811 00:26:28.676856 8409 consensus_queue.cc:237] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250108779aeb4d4fadcb6aae04f5ca73" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 39345 } } peers { permanent_uuid: "8de08b4f4047467186e7181ef1252d0f" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 35461 } } peers { permanent_uuid: "a199d558849b4074a1513877d7f71043" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44811 } }
I20260811 00:26:28.683316 7878 catalog_manager.cc:5495] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 reported cstate change: term changed from 0 to 1, leader changed from <none> to a199d558849b4074a1513877d7f71043 (127.2.120.1). New cstate: current_term: 1 leader_uuid: "a199d558849b4074a1513877d7f71043" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "250108779aeb4d4fadcb6aae04f5ca73" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 39345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8de08b4f4047467186e7181ef1252d0f" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 35461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a199d558849b4074a1513877d7f71043" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44811 } health_report { overall_health: HEALTHY } } }
I20260811 00:26:28.735932 8400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:28.733657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33050 (local address 127.2.120.1:44811)
0811 00:26:28.733884 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:28.733892 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:28.733916 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:26:28.734059 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:28.734064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:28.734170 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:28.734280 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:28.735269 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:28.735273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:28.735277 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:28.735292 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:28.735323 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:28.735335 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:28.735436 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:28.735554 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 00:26:28.735601 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 00:26:28.745226 8331 raft_consensus.cc:1261] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73 [term 1 FOLLOWER]: Refusing update from remote peer a199d558849b4074a1513877d7f71043: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:28.745261 8185 raft_consensus.cc:1261] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f [term 1 FOLLOWER]: Refusing update from remote peer a199d558849b4074a1513877d7f71043: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:28.746385 8414 consensus_queue.cc:1035] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [LEADER]: Connected to new peer: Peer: permanent_uuid: "250108779aeb4d4fadcb6aae04f5ca73" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 39345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:28.746798 8409 consensus_queue.cc:1035] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8de08b4f4047467186e7181ef1252d0f" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 35461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:28.787397 7880 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:28.787722 7880 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:28.793121 7880 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21a2be276fb24d72a683fe1105116fae: Sending DeleteTablet for 3 replicas of tablet d14c8023df7e49bb9338a47791ef1942
I20260811 00:26:28.794680 8160 tablet_service.cc:1514] Processing DeleteTablet for tablet d14c8023df7e49bb9338a47791ef1942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:28 UTC) from {username='slave'} at 127.0.0.1:52698
I20260811 00:26:28.795109 8016 tablet_service.cc:1514] Processing DeleteTablet for tablet d14c8023df7e49bb9338a47791ef1942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:28 UTC) from {username='slave'} at 127.0.0.1:33034
I20260811 00:26:28.795908 8309 tablet_service.cc:1514] Processing DeleteTablet for tablet d14c8023df7e49bb9338a47791ef1942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:28 UTC) from {username='slave'} at 127.0.0.1:41886
I20260811 00:26:28.796336 8431 tablet_replica.cc:330] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043: stopping tablet replica
I20260811 00:26:28.796411 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 7945
I20260811 00:26:28.797013 8431 raft_consensus.cc:2229] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:26:28.797986 8431 raft_consensus.cc:2258] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:28.798997 8432 tablet_replica.cc:330] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73: stopping tablet replica
I20260811 00:26:28.799341 8430 tablet_replica.cc:330] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f: stopping tablet replica
I20260811 00:26:28.799643 8432 raft_consensus.cc:2229] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:28.799763 8430 raft_consensus.cc:2229] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:28.800307 8432 raft_consensus.cc:2258] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:28.800307 8430 raft_consensus.cc:2258] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:28.801954 8431 ts_tablet_manager.cc:1905] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:28.802644 8430 ts_tablet_manager.cc:1905] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:28.802660 8432 ts_tablet_manager.cc:1905] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:28.810724 8432 ts_tablet_manager.cc:1918] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:26:28.810724 8430 ts_tablet_manager.cc:1918] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:26:28.810932 8430 log.cc:1196] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/d14c8023df7e49bb9338a47791ef1942
I20260811 00:26:28.811120 8432 log.cc:1196] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/d14c8023df7e49bb9338a47791ef1942
I20260811 00:26:28.811439 8431 ts_tablet_manager.cc:1918] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:26:28.811599 8430 ts_tablet_manager.cc:1939] T d14c8023df7e49bb9338a47791ef1942 P 8de08b4f4047467186e7181ef1252d0f: Deleting consensus metadata
I20260811 00:26:28.811690 8431 log.cc:1196] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/d14c8023df7e49bb9338a47791ef1942
I20260811 00:26:28.812022 8432 ts_tablet_manager.cc:1939] T d14c8023df7e49bb9338a47791ef1942 P 250108779aeb4d4fadcb6aae04f5ca73: Deleting consensus metadata
I20260811 00:26:28.812397 8431 ts_tablet_manager.cc:1939] T d14c8023df7e49bb9338a47791ef1942 P a199d558849b4074a1513877d7f71043: Deleting consensus metadata
I20260811 00:26:28.814430 7862 catalog_manager.cc:4841] TS 8de08b4f4047467186e7181ef1252d0f (127.2.120.2:35461): tablet d14c8023df7e49bb9338a47791ef1942 (table test-table [id=370f18175dbc4559b022a1368e12a662]) successfully deleted
I20260811 00:26:28.814863 7862 catalog_manager.cc:4841] TS 250108779aeb4d4fadcb6aae04f5ca73 (127.2.120.3:39345): tablet d14c8023df7e49bb9338a47791ef1942 (table test-table [id=370f18175dbc4559b022a1368e12a662]) successfully deleted
I20260811 00:26:28.814455 7863 catalog_manager.cc:4841] TS a199d558849b4074a1513877d7f71043 (127.2.120.1:44811): tablet d14c8023df7e49bb9338a47791ef1942 (table test-table [id=370f18175dbc4559b022a1368e12a662]) successfully deleted
I20260811 00:26:28.817094 8400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:28.798474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33060 (local address 127.2.120.1:44811)
0811 00:26:28.798707 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:28.798713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:28.813659 (+ 14946us) server_negotiation.cc:407] Connection header received
0811 00:26:28.814814 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:28.814821 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:28.814892 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:28.814973 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:28.816231 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:28.816236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:28.816249 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:28.816268 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:28.816309 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:28.816322 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:28.816475 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:28.816657 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 00:26:28.816715 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 00:26:28.817888 8085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:28.922493 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 7945
I20260811 00:26:28.945748 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 8094
I20260811 00:26:28.949318 8402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:28.946668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52702 (local address 127.2.120.2:35461)
0811 00:26:28.946956 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:28.946961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:28.946981 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:26:28.947042 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:28.947046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:28.947151 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:28.947255 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:28.948386 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:28.948393 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:28.948399 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:28.948420 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:28.948465 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:28.948479 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:28.948677 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:28.948803 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 00:26:28.948872 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 00:26:28.950135 8229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:29.046388 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 8094
I20260811 00:26:29.068470 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 8238
I20260811 00:26:29.072304 8397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:29.069586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41896 (local address 127.2.120.3:39345)
0811 00:26:29.069851 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:29.069859 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:29.070025 (+ 166us) server_negotiation.cc:407] Connection header received
0811 00:26:29.070248 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:29.070253 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:29.070343 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:29.070445 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:29.071544 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:29.071548 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:29.071552 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:29.071568 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:29.071603 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:29.071615 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:29.071734 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:29.071866 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 00:26:29.071963 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260811 00:26:29.073029 8376 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:29.164682 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 8238
I20260811 00:26:29.187542 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 7843
I20260811 00:26:29.188984 7914 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:29.298388 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 7843
2026-08-11T00:26:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3174 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 00:26:29.346589 2528 mini_oidc.cc:152] Starting JWKS server
I20260811 00:26:29.351465 2528 webserver.cc:460] Webserver started at https://127.0.0.1:44229/ using document root <none> and password file <none>
I20260811 00:26:29.351928 2528 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 00:26:29.352823 2528 webserver.cc:460] Webserver started at http://127.0.0.1:40311/ using document root <none> and password file <none>
2026-08-11T00:26:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:26:29Z Disabled control of system clock
I20260811 00:26:29.389567 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:40603
--webserver_interface=127.2.120.62
--webserver_port=0
--builtin_ntp_servers=127.2.120.20:39941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:40603
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44229/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/testchainca.cert with env {}
W20260811 00:26:29.757470 8445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:29.757936 8445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:29.758129 8445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:29.766943 8445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:26:29.767150 8445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:29.767251 8445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:29.767364 8445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:26:29.767442 8445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 00:26:29.767536 8445 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 00:26:29.767619 8445 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44229/jwks.json
I20260811 00:26:29.780845 8445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:39941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:40603
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:40603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44229/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:29.783288 8445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:29.786034 8445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:29.809074 8452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:29.809074 8455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:29.808913 8453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:29.811642 8445 server_base.cc:1034] running on GCE node
I20260811 00:26:29.813477 8445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:29.816987 8445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:29.819008 8445 hybrid_clock.cc:648] HybridClock initialized: now 1786407989818853 us; error 113 us; skew 500 ppm
I20260811 00:26:29.819908 8445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:26:29.836165 8445 webserver.cc:460] Webserver started at http://127.2.120.62:40255/ using document root <none> and password file <none>
I20260811 00:26:29.837167 8445 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:29.837286 8445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:29.837687 8445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:29.841063 8445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "83ed5007f6344e219dc83b36b6615996"
format_stamp: "Formatted at 2026-08-11 00:26:29 on dist-test-slave-fd7l"
server_key: "dcbbeac4cfa4c234f704d4c8e46b0a2c"
server_key_iv: "819fc25238b5e69be9cb61572d1db579"
server_key_version: "encryptionkey@0"
I20260811 00:26:29.842005 8445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "83ed5007f6344e219dc83b36b6615996"
format_stamp: "Formatted at 2026-08-11 00:26:29 on dist-test-slave-fd7l"
server_key: "dcbbeac4cfa4c234f704d4c8e46b0a2c"
server_key_iv: "819fc25238b5e69be9cb61572d1db579"
server_key_version: "encryptionkey@0"
I20260811 00:26:29.848111 8445 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 00:26:29.852211 8462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:29.854122 8445 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:26:29.854372 8445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "83ed5007f6344e219dc83b36b6615996"
format_stamp: "Formatted at 2026-08-11 00:26:29 on dist-test-slave-fd7l"
server_key: "dcbbeac4cfa4c234f704d4c8e46b0a2c"
server_key_iv: "819fc25238b5e69be9cb61572d1db579"
server_key_version: "encryptionkey@0"
I20260811 00:26:29.854663 8445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:29.877246 8445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:29.878926 8445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:29.879411 8445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:29.905627 8445 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:40603
I20260811 00:26:29.905660 8520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:40603 every 8 connection(s)
I20260811 00:26:29.907269 8445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:26:29.908080 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 8445
I20260811 00:26:29.908465 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:26:29.909571 2528 external_mini_cluster.cc:1428] Setting key f691c0eee58ee81edd2efee2ce412006
I20260811 00:26:29.913043 8521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:29.930249 8524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:29.912224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50016 (local address 127.2.120.62:40603)
0811 00:26:29.913798 (+ 1574us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:29.913816 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:29.913882 (+ 66us) server_negotiation.cc:407] Connection header received
0811 00:26:29.916775 (+ 2893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:29.916813 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:29.917671 (+ 858us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:29.918292 (+ 621us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:29.918998 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:29.921557 (+ 2559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:29.923615 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:29.924892 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:29.925141 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:29.926616 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:29.926632 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:29.926661 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:29.927778 (+ 1117us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:29.927893 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:29.927931 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:29.928080 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:29.928399 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 00:26:29.928830 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":923,"thread_start_us":634,"threads_started":1}
I20260811 00:26:29.930588 8521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996: Bootstrap starting.
I20260811 00:26:29.934705 8521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:29.935748 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:26:29.936087 8521 log.cc:824] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:29.939932 8521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996: No bootstrap required, opened a new log
I20260811 00:26:29.953500 8521 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ed5007f6344e219dc83b36b6615996" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40603 } }
I20260811 00:26:29.954090 8521 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:29.954244 8521 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ed5007f6344e219dc83b36b6615996, State: Initialized, Role: FOLLOWER
I20260811 00:26:29.954988 8521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ed5007f6344e219dc83b36b6615996" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40603 } }
I20260811 00:26:29.955241 8521 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:26:29.955430 8521 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:26:29.955657 8521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:29.959167 8521 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ed5007f6344e219dc83b36b6615996" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40603 } }
I20260811 00:26:29.960124 8521 leader_election.cc:302] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83ed5007f6344e219dc83b36b6615996; no voters:
I20260811 00:26:29.961107 8521 leader_election.cc:288] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:26:29.961390 8526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:29.962450 8526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996 [term 1 LEADER]: Becoming Leader. State: Replica: 83ed5007f6344e219dc83b36b6615996, State: Running, Role: LEADER
I20260811 00:26:29.963168 8526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ed5007f6344e219dc83b36b6615996" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40603 } }
I20260811 00:26:29.964197 8521 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:26:29.966897 8527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83ed5007f6344e219dc83b36b6615996" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ed5007f6344e219dc83b36b6615996" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40603 } } }
I20260811 00:26:29.967504 8527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996 [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:29.969146 8528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83ed5007f6344e219dc83b36b6615996. Latest consensus state: current_term: 1 leader_uuid: "83ed5007f6344e219dc83b36b6615996" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ed5007f6344e219dc83b36b6615996" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40603 } } }
I20260811 00:26:29.969594 8528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996 [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:29.970074 8532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:26:29.977238 8532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:26:29.990260 8532 catalog_manager.cc:1347] Generated new cluster ID: bc630131a1584ea196127cc84625de93
I20260811 00:26:29.990424 8532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:26:30.011725 8532 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:26:30.013864 8532 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:26:30.028430 8532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83ed5007f6344e219dc83b36b6615996: Generated new TSK 0
I20260811 00:26:30.029831 8532 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 00:26:30.083127 8555 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 00:26:30.083393 8555 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 00:26:30.085880 8524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:30.057228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50032 (local address 127.2.120.62:40603)
0811 00:26:30.057600 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:30.057609 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:30.058691 (+ 1082us) server_negotiation.cc:407] Connection header received
0811 00:26:30.059353 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:30.059359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:30.059466 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:30.059870 (+ 404us) server_negotiation.cc:226] Negotiated authn=JWT
0811 00:26:30.060787 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:30.061765 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:30.063708 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:30.064426 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:30.064674 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:30.065843 (+ 1169us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 00:26:30.085060 (+ 19217us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 00:26:30.085518 (+ 458us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50032: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":176}
W20260811 00:26:30.086028 8524 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50032: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 00:26:30.087580 8554 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:26:30.057200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:40603 (local address 127.0.0.1:50032)
0811 00:26:30.058497 (+ 1297us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:30.058522 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:30.058847 (+ 325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:30.059890 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:30.059898 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:30.059924 (+ 26us) client_negotiation.cc:189] Negotiated authn=JWT
0811 00:26:30.060421 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:30.060434 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:30.061995 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:30.062001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:30.063467 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:30.063484 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:30.064739 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:30.064748 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:30.065046 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:30.065164 (+ 118us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 00:26:30.085465 (+ 20301us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 00:26:30.085791 (+ 326us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.120.62:40603: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":987,"thread_start_us":774,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.2.120.62:40603: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 00:26:30.089687 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 8445
I20260811 00:26:30.091130 8516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:30.190794 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 8445
2026-08-11T00:26:30Z chronyd exiting
I20260811 00:26:30.250628 2528 test_util.cc:175] -----------------------------------------------
I20260811 00:26:30.250808 2528 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407939260739-2528-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (909 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 00:26:30.255175 2528 mini_oidc.cc:152] Starting JWKS server
I20260811 00:26:30.257277 2528 webserver.cc:460] Webserver started at https://127.0.0.1:39269/ using document root <none> and password file <none>
I20260811 00:26:30.257540 2528 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 00:26:30.258282 2528 webserver.cc:460] Webserver started at http://127.0.0.1:33305/ using document root <none> and password file <none>
2026-08-11T00:26:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:26:30Z Disabled control of system clock
I20260811 00:26:30.292670 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:40655
--webserver_interface=127.2.120.62
--webserver_port=0
--builtin_ntp_servers=127.2.120.20:34087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:40655
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39269/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/testchainca.cert with env {}
W20260811 00:26:30.671399 8564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:30.671838 8564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:30.672019 8564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:30.680475 8564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:26:30.680995 8564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:30.681165 8564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:30.681289 8564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:26:30.681383 8564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 00:26:30.681484 8564 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 00:26:30.681568 8564 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39269/jwks.json
I20260811 00:26:30.693640 8564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:34087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:40655
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:40655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39269/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:30.695324 8564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:30.696988 8564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:30.710928 8569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:30.710855 8570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:30.710871 8572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:30.711997 8564 server_base.cc:1034] running on GCE node
I20260811 00:26:30.713091 8564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:30.714781 8564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:30.716058 8564 hybrid_clock.cc:648] HybridClock initialized: now 1786407990715997 us; error 58 us; skew 500 ppm
I20260811 00:26:30.716488 8564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:26:30.719309 8564 webserver.cc:460] Webserver started at http://127.2.120.62:34597/ using document root <none> and password file <none>
I20260811 00:26:30.720153 8564 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:30.720314 8564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:30.720775 8564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:30.724102 8564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "aab724421cf249dea12584e4243ca36a"
format_stamp: "Formatted at 2026-08-11 00:26:30 on dist-test-slave-fd7l"
server_key: "d231706e514070d8e3b0fc61762d57e3"
server_key_iv: "20278a8261b5f85eea2a1341f01466d8"
server_key_version: "encryptionkey@0"
I20260811 00:26:30.725126 8564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "aab724421cf249dea12584e4243ca36a"
format_stamp: "Formatted at 2026-08-11 00:26:30 on dist-test-slave-fd7l"
server_key: "d231706e514070d8e3b0fc61762d57e3"
server_key_iv: "20278a8261b5f85eea2a1341f01466d8"
server_key_version: "encryptionkey@0"
I20260811 00:26:30.731317 8564 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:26:30.735755 8584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:30.737658 8564 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 00:26:30.737910 8564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "aab724421cf249dea12584e4243ca36a"
format_stamp: "Formatted at 2026-08-11 00:26:30 on dist-test-slave-fd7l"
server_key: "d231706e514070d8e3b0fc61762d57e3"
server_key_iv: "20278a8261b5f85eea2a1341f01466d8"
server_key_version: "encryptionkey@0"
I20260811 00:26:30.738197 8564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:30.756850 8564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:30.758224 8564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:30.758733 8564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:30.782117 8564 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:40655
I20260811 00:26:30.782152 8640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:40655 every 8 connection(s)
I20260811 00:26:30.783764 8564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:26:30.788292 8642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:30.789068 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 8564
I20260811 00:26:30.789310 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:26:30.790215 2528 external_mini_cluster.cc:1428] Setting key f81b5a447b6a5af2c99ad64b5c077dc9
I20260811 00:26:30.802949 8642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a: Bootstrap starting.
I20260811 00:26:30.805617 8645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:30.792639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50382 (local address 127.2.120.62:40655)
0811 00:26:30.793850 (+ 1211us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:30.793863 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:30.793916 (+ 53us) server_negotiation.cc:407] Connection header received
0811 00:26:30.795880 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:30.795910 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:30.796599 (+ 689us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:30.797111 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:30.797767 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:30.799152 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:30.800639 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:30.801397 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:30.801638 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:30.802983 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:30.802993 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:30.803010 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:30.803655 (+ 645us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:30.803745 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:30.803769 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:30.803912 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:30.804168 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 00:26:30.804574 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":427,"threads_started":1}
I20260811 00:26:30.806958 8642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:30.808143 8642 log.cc:824] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:30.810052 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:26:30.811414 8642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a: No bootstrap required, opened a new log
I20260811 00:26:30.819164 8642 raft_consensus.cc:348] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aab724421cf249dea12584e4243ca36a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40655 } }
I20260811 00:26:30.819607 8642 raft_consensus.cc:374] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:30.819713 8642 raft_consensus.cc:729] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aab724421cf249dea12584e4243ca36a, State: Initialized, Role: FOLLOWER
I20260811 00:26:30.820699 8642 consensus_queue.cc:260] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aab724421cf249dea12584e4243ca36a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40655 } }
I20260811 00:26:30.821002 8642 raft_consensus.cc:388] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:26:30.821163 8642 raft_consensus.cc:482] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:26:30.821370 8642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:30.824661 8642 raft_consensus.cc:504] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aab724421cf249dea12584e4243ca36a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40655 } }
I20260811 00:26:30.825237 8642 leader_election.cc:302] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aab724421cf249dea12584e4243ca36a; no voters:
I20260811 00:26:30.825894 8642 leader_election.cc:288] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:26:30.826107 8647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:30.827013 8647 raft_consensus.cc:686] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a [term 1 LEADER]: Becoming Leader. State: Replica: aab724421cf249dea12584e4243ca36a, State: Running, Role: LEADER
I20260811 00:26:30.827553 8647 consensus_queue.cc:237] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aab724421cf249dea12584e4243ca36a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40655 } }
I20260811 00:26:30.828397 8642 sys_catalog.cc:564] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:26:30.830735 8650 sys_catalog.cc:455] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a [sys.catalog]: SysCatalogTable state changed. Reason: New leader aab724421cf249dea12584e4243ca36a. Latest consensus state: current_term: 1 leader_uuid: "aab724421cf249dea12584e4243ca36a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aab724421cf249dea12584e4243ca36a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40655 } } }
I20260811 00:26:30.831269 8650 sys_catalog.cc:458] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:30.832348 8648 sys_catalog.cc:455] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aab724421cf249dea12584e4243ca36a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aab724421cf249dea12584e4243ca36a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40655 } } }
I20260811 00:26:30.832813 8648 sys_catalog.cc:458] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:30.834342 8655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:26:30.839957 8655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:26:30.848994 8655 catalog_manager.cc:1347] Generated new cluster ID: 05739819c6d54a6a8e2bb86f572ceadf
I20260811 00:26:30.849135 8655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:26:30.859784 8655 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:26:30.861769 8655 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:26:30.869688 8655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aab724421cf249dea12584e4243ca36a: Generated new TSK 0
I20260811 00:26:30.870882 8655 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 00:26:30.910852 8673 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 00:26:30.911013 8673 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 00:26:30.912302 8645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:30.890725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50388 (local address 127.2.120.62:40655)
0811 00:26:30.890922 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:30.890927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:30.891478 (+ 551us) server_negotiation.cc:407] Connection header received
0811 00:26:30.891759 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:30.891767 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:30.891869 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:30.892081 (+ 212us) server_negotiation.cc:226] Negotiated authn=JWT
0811 00:26:30.892599 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:30.893766 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:30.895376 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:30.896090 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:30.896236 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:30.896886 (+ 650us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 00:26:30.911537 (+ 14651us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39269/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 00:26:30.911976 (+ 439us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50388: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39269/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 00:26:30.912431 8645 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50388: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39269/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 00:26:30.912562 8672 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:26:30.890571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:40655 (local address 127.0.0.1:50388)
0811 00:26:30.891355 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:30.891371 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:30.891513 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:30.892112 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:30.892116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:30.892127 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0811 00:26:30.892397 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:30.892405 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:30.893954 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:30.893959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:30.895214 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:30.895226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:30.896314 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:30.896320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:30.896465 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:30.896504 (+ 39us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 00:26:30.911915 (+ 15411us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39269/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 00:26:30.912154 (+ 239us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.120.62:40655: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39269/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":484,"threads_started":1}
I20260811 00:26:30.913825 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 8564
I20260811 00:26:30.915028 8636 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:31.008174 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 8564
2026-08-11T00:26:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (814 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 00:26:31.069383 2528 mini_oidc.cc:152] Starting JWKS server
I20260811 00:26:31.071568 2528 webserver.cc:460] Webserver started at https://127.0.0.1:35077/ using document root <none> and password file <none>
I20260811 00:26:31.071875 2528 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 00:26:31.072832 2528 webserver.cc:460] Webserver started at http://127.0.0.1:38407/ using document root <none> and password file <none>
2026-08-11T00:26:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:26:31Z Disabled control of system clock
I20260811 00:26:31.113732 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:44413
--webserver_interface=127.2.120.62
--webserver_port=0
--builtin_ntp_servers=127.2.120.20:45005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:44413
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35077/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 00:26:31.506565 8681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:31.506986 8681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:31.507156 8681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:31.514725 8681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:26:31.514909 8681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:31.515004 8681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:31.515125 8681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:26:31.515203 8681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 00:26:31.515293 8681 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 00:26:31.515372 8681 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35077/jwks.json
I20260811 00:26:31.524957 8681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:45005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:44413
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:44413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35077/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:31.526394 8681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:31.528348 8681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:31.541709 8686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:31.541674 8688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:31.541630 8692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:31.542467 8681 server_base.cc:1034] running on GCE node
I20260811 00:26:31.543385 8681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:31.545125 8681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:31.546370 8681 hybrid_clock.cc:648] HybridClock initialized: now 1786407991546316 us; error 56 us; skew 500 ppm
I20260811 00:26:31.546774 8681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:26:31.549526 8681 webserver.cc:460] Webserver started at http://127.2.120.62:45907/ using document root <none> and password file <none>
I20260811 00:26:31.550316 8681 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:31.550458 8681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:31.550810 8681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:31.554036 8681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "b7a85bda43ee41c9a1d28a012cb4647b"
format_stamp: "Formatted at 2026-08-11 00:26:31 on dist-test-slave-fd7l"
server_key: "cc7d1ad33301182f0dfc3e0efe0b35b5"
server_key_iv: "aa245ba1ed820ed82d1c5e9fe15cdc68"
server_key_version: "encryptionkey@0"
I20260811 00:26:31.555018 8681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "b7a85bda43ee41c9a1d28a012cb4647b"
format_stamp: "Formatted at 2026-08-11 00:26:31 on dist-test-slave-fd7l"
server_key: "cc7d1ad33301182f0dfc3e0efe0b35b5"
server_key_iv: "aa245ba1ed820ed82d1c5e9fe15cdc68"
server_key_version: "encryptionkey@0"
I20260811 00:26:31.561606 8681 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 00:26:31.566196 8698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:31.567982 8681 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260811 00:26:31.568239 8681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "b7a85bda43ee41c9a1d28a012cb4647b"
format_stamp: "Formatted at 2026-08-11 00:26:31 on dist-test-slave-fd7l"
server_key: "cc7d1ad33301182f0dfc3e0efe0b35b5"
server_key_iv: "aa245ba1ed820ed82d1c5e9fe15cdc68"
server_key_version: "encryptionkey@0"
I20260811 00:26:31.568516 8681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:31.601408 8681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:31.602452 8681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:31.602829 8681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:31.625192 8681 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:44413
I20260811 00:26:31.625200 8756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:44413 every 8 connection(s)
I20260811 00:26:31.626763 8681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:26:31.631112 8757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:31.632844 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 8681
I20260811 00:26:31.633106 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:26:31.633975 2528 external_mini_cluster.cc:1428] Setting key e65730f9192b320527d61424d4211f9f
I20260811 00:26:31.647261 8757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b: Bootstrap starting.
I20260811 00:26:31.651767 8757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:31.653349 8757 log.cc:824] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:31.654426 8760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:31.636978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54120 (local address 127.2.120.62:44413)
0811 00:26:31.638697 (+ 1719us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:31.638715 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:31.638786 (+ 71us) server_negotiation.cc:407] Connection header received
0811 00:26:31.641412 (+ 2626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:31.641452 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:31.642311 (+ 859us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:31.642911 (+ 600us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:31.643802 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:31.646189 (+ 2387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:31.648029 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:31.649208 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:31.649464 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:31.650809 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:31.650821 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:31.650843 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:31.651630 (+ 787us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:31.651778 (+ 148us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:31.651809 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:31.651942 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:31.652322 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 00:26:31.652743 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1029,"thread_start_us":600,"threads_started":1}
I20260811 00:26:31.659273 8757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b: No bootstrap required, opened a new log
I20260811 00:26:31.660739 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:26:31.670439 8757 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7a85bda43ee41c9a1d28a012cb4647b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44413 } }
I20260811 00:26:31.671154 8757 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:31.671283 8757 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7a85bda43ee41c9a1d28a012cb4647b, State: Initialized, Role: FOLLOWER
I20260811 00:26:31.671974 8757 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7a85bda43ee41c9a1d28a012cb4647b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44413 } }
I20260811 00:26:31.672297 8757 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:26:31.672462 8757 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:26:31.672706 8757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:31.675923 8757 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7a85bda43ee41c9a1d28a012cb4647b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44413 } }
I20260811 00:26:31.676662 8757 leader_election.cc:302] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7a85bda43ee41c9a1d28a012cb4647b; no voters:
I20260811 00:26:31.677417 8757 leader_election.cc:288] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:26:31.677904 8762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:31.679399 8762 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b [term 1 LEADER]: Becoming Leader. State: Replica: b7a85bda43ee41c9a1d28a012cb4647b, State: Running, Role: LEADER
I20260811 00:26:31.680065 8762 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7a85bda43ee41c9a1d28a012cb4647b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44413 } }
I20260811 00:26:31.680678 8757 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:26:31.683764 8763 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7a85bda43ee41c9a1d28a012cb4647b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7a85bda43ee41c9a1d28a012cb4647b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44413 } } }
I20260811 00:26:31.684229 8763 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:31.683647 8764 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7a85bda43ee41c9a1d28a012cb4647b. Latest consensus state: current_term: 1 leader_uuid: "b7a85bda43ee41c9a1d28a012cb4647b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7a85bda43ee41c9a1d28a012cb4647b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44413 } } }
I20260811 00:26:31.684583 8764 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:31.686673 8771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:26:31.692796 8771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:26:31.705842 8771 catalog_manager.cc:1347] Generated new cluster ID: 178250cb18ce4d80ba4e60bbd72ea520
I20260811 00:26:31.705989 8771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:26:31.746960 8771 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:26:31.748880 8771 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:26:31.762147 8771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7a85bda43ee41c9a1d28a012cb4647b: Generated new TSK 0
I20260811 00:26:31.763214 8771 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:26:31.794643 8760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:31.785683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54132 (local address 127.2.120.62:44413)
0811 00:26:31.785967 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:31.785974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:31.786365 (+ 391us) server_negotiation.cc:407] Connection header received
0811 00:26:31.786665 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:31.786670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:31.786744 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:31.786886 (+ 142us) server_negotiation.cc:226] Negotiated authn=JWT
0811 00:26:31.787451 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:31.788183 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:31.790329 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:31.790843 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:31.790951 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:31.791695 (+ 744us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 00:26:31.793909 (+ 2214us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35077/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 00:26:31.794305 (+ 396us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54132: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35077/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":97}
W20260811 00:26:31.794799 8760 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54132: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35077/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 00:26:31.794900 8791 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:26:31.785552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44413 (local address 127.0.0.1:54132)
0811 00:26:31.786242 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:31.786263 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:31.786437 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:31.786940 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:31.786946 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:31.786960 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0811 00:26:31.787288 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:31.787301 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:31.788340 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:31.788346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:31.790142 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:31.790160 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:31.791067 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:31.791074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:31.791288 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:31.791340 (+ 52us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 00:26:31.794151 (+ 2811us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35077/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 00:26:31.794445 (+ 294us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.120.62:44413: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35077/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":380,"threads_started":1}
I20260811 00:26:31.795831 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 8681
I20260811 00:26:31.797309 8751 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 00:26:31.812664 8792 webserver.cc:533] Webserver: sslize failed:
E20260811 00:26:31.812811 8792 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 00:26:31.895402 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 8681
2026-08-11T00:26:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (891 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T00:26:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:26:31Z Disabled control of system clock
I20260811 00:26:31.999048 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:39963
--webserver_interface=127.2.120.62
--webserver_port=0
--builtin_ntp_servers=127.2.120.20:38693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:39963
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/testchainca.cert with env {}
W20260811 00:26:32.361330 8797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:32.361685 8797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:32.361776 8797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:32.369581 8797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:26:32.369730 8797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:32.369771 8797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:32.369825 8797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:26:32.369863 8797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 00:26:32.369921 8797 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 00:26:32.369977 8797 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 00:26:32.380625 8797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:38693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:39963
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:39963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:32.382066 8797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:32.383430 8797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:32.396733 8803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:32.396649 8805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:32.396775 8802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:32.397907 8797 server_base.cc:1034] running on GCE node
I20260811 00:26:32.399044 8797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:32.400847 8797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:32.402161 8797 hybrid_clock.cc:648] HybridClock initialized: now 1786407992402125 us; error 54 us; skew 500 ppm
I20260811 00:26:32.402623 8797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:26:32.405732 8797 webserver.cc:460] Webserver started at http://127.2.120.62:46845/ using document root <none> and password file <none>
I20260811 00:26:32.406587 8797 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:32.406749 8797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:32.407236 8797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:32.410668 8797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "582a53ef6d164c12b8e6f18432b78ae0"
format_stamp: "Formatted at 2026-08-11 00:26:32 on dist-test-slave-fd7l"
server_key: "0402629f368de491e2b94bbb6778f530"
server_key_iv: "e567500db9c27ba903e85852720e9abb"
server_key_version: "encryptionkey@0"
I20260811 00:26:32.411603 8797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "582a53ef6d164c12b8e6f18432b78ae0"
format_stamp: "Formatted at 2026-08-11 00:26:32 on dist-test-slave-fd7l"
server_key: "0402629f368de491e2b94bbb6778f530"
server_key_iv: "e567500db9c27ba903e85852720e9abb"
server_key_version: "encryptionkey@0"
I20260811 00:26:32.417491 8797 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 00:26:32.422060 8811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:32.423633 8797 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:26:32.423851 8797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "582a53ef6d164c12b8e6f18432b78ae0"
format_stamp: "Formatted at 2026-08-11 00:26:32 on dist-test-slave-fd7l"
server_key: "0402629f368de491e2b94bbb6778f530"
server_key_iv: "e567500db9c27ba903e85852720e9abb"
server_key_version: "encryptionkey@0"
I20260811 00:26:32.424134 8797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:32.437019 8797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:32.437860 8797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:32.438114 8797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:32.460711 8869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:39963 every 8 connection(s)
I20260811 00:26:32.460731 8797 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:39963
I20260811 00:26:32.462530 8797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:26:32.465289 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 8797
I20260811 00:26:32.465513 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:26:32.466219 2528 external_mini_cluster.cc:1428] Setting key 2e2848b51ca7cebbc89361914d52df1a
I20260811 00:26:32.468397 8870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:32.480794 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:32.468337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60004 (local address 127.2.120.62:39963)
0811 00:26:32.469462 (+ 1125us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:32.469475 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:32.469518 (+ 43us) server_negotiation.cc:407] Connection header received
0811 00:26:32.471157 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:32.471179 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:32.471676 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:32.472112 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:32.472779 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:32.474183 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:32.475580 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:32.476353 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:32.476618 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:32.478052 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:32.478061 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:32.478079 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:26:32.478726 (+ 647us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:32.478834 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:26:32.478861 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:32.479023 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:32.479295 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 00:26:32.479667 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":386,"threads_started":1}
I20260811 00:26:32.482163 8870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0: Bootstrap starting.
I20260811 00:26:32.485144 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:26:32.486101 8870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:32.487279 8870 log.cc:824] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:32.491307 8870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0: No bootstrap required, opened a new log
I20260811 00:26:32.499575 8870 raft_consensus.cc:348] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "582a53ef6d164c12b8e6f18432b78ae0" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39963 } }
I20260811 00:26:32.500130 8870 raft_consensus.cc:374] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:32.500264 8870 raft_consensus.cc:729] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 582a53ef6d164c12b8e6f18432b78ae0, State: Initialized, Role: FOLLOWER
I20260811 00:26:32.501160 8870 consensus_queue.cc:260] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "582a53ef6d164c12b8e6f18432b78ae0" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39963 } }
I20260811 00:26:32.501515 8870 raft_consensus.cc:388] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:26:32.501708 8870 raft_consensus.cc:482] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:26:32.501926 8870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:32.505347 8870 raft_consensus.cc:504] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "582a53ef6d164c12b8e6f18432b78ae0" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39963 } }
I20260811 00:26:32.505954 8870 leader_election.cc:302] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 582a53ef6d164c12b8e6f18432b78ae0; no voters:
I20260811 00:26:32.506623 8870 leader_election.cc:288] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:26:32.506778 8876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:32.507619 8876 raft_consensus.cc:686] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0 [term 1 LEADER]: Becoming Leader. State: Replica: 582a53ef6d164c12b8e6f18432b78ae0, State: Running, Role: LEADER
I20260811 00:26:32.508173 8876 consensus_queue.cc:237] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "582a53ef6d164c12b8e6f18432b78ae0" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39963 } }
I20260811 00:26:32.509126 8870 sys_catalog.cc:564] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:26:32.511842 8878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 582a53ef6d164c12b8e6f18432b78ae0. Latest consensus state: current_term: 1 leader_uuid: "582a53ef6d164c12b8e6f18432b78ae0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "582a53ef6d164c12b8e6f18432b78ae0" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39963 } } }
I20260811 00:26:32.512267 8878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0 [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:32.512765 8877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "582a53ef6d164c12b8e6f18432b78ae0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "582a53ef6d164c12b8e6f18432b78ae0" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39963 } } }
I20260811 00:26:32.513108 8877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0 [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:32.518442 8885 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:26:32.524899 8885 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:26:32.536157 8885 catalog_manager.cc:1347] Generated new cluster ID: 3d3a7b7d134c4c459efba13af9e9b645
I20260811 00:26:32.536321 8885 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:26:32.547578 8885 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:26:32.549646 8885 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:26:32.559684 8885 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 582a53ef6d164c12b8e6f18432b78ae0: Generated new TSK 0
I20260811 00:26:32.560957 8885 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:26:33.079299 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:32.579247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60016 (local address 127.2.120.62:39963)
0811 00:26:32.579604 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:32.579616 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:32.580132 (+ 516us) server_negotiation.cc:407] Connection header received
0811 00:26:32.580663 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:32.580671 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:32.580789 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:32.581015 (+ 226us) server_negotiation.cc:226] Negotiated authn=JWT
0811 00:26:32.581957 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:32.583116 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:32.586308 (+ 3192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:32.587117 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:32.587347 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:32.588799 (+ 1452us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 00:26:33.078664 (+489865us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 00:26:33.079051 (+ 387us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60016: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":115}
W20260811 00:26:33.079473 8873 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60016: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 00:26:33.079702 8902 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:26:32.579038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:39963 (local address 127.0.0.1:60016)
0811 00:26:32.579922 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:32.579950 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:32.580240 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:32.581114 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:32.581123 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:32.581145 (+ 22us) client_negotiation.cc:189] Negotiated authn=JWT
0811 00:26:32.581677 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:32.581696 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:32.583457 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:32.583466 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:32.585762 (+ 2296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:32.585801 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:32.587564 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:32.587576 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:32.587864 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:32.587964 (+ 100us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 00:26:33.079001 (+491037us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 00:26:33.079287 (+ 286us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.120.62:39963: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":475,"threads_started":1}
I20260811 00:26:33.080952 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 8797
I20260811 00:26:33.082370 8864 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:33.192691 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 8797
2026-08-11T00:26:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1278 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (477 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (450 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:26:34 dist-test-slave-fd7l krb5kdc[8917](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:26:34 dist-test-slave-fd7l krb5kdc[8917](info): set up 2 sockets
Aug 11 00:26:34 dist-test-slave-fd7l krb5kdc[8917](info): commencing operation
krb5kdc: starting...
W20260811 00:26:35.364480 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.006s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:26:35 dist-test-slave-fd7l krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:26:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:26:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:26:35.646250 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:34903
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:46281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:34903
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:36.024072 8932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:36.024427 8932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:36.024593 8932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:36.032162 8932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:26:36.032291 8932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:36.032339 8932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:36.032389 8932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:26:36.032464 8932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:26:36.042917 8932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:46281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:34903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:34903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:36.044569 8932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:36.046125 8932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:36.060721 8940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:36.060674 8938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:36.060770 8937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:36.067775 8932 server_base.cc:1034] running on GCE node
I20260811 00:26:36.069305 8932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:36.071311 8932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:36.072693 8932 hybrid_clock.cc:648] HybridClock initialized: now 1786407996072638 us; error 51 us; skew 500 ppm
I20260811 00:26:36.073239 8932 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:36 dist-test-slave-fd7l krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407996, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:36.081672 8932 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:26:36.083976 8932 webserver.cc:460] Webserver started at http://127.2.120.62:38763/ using document root <none> and password file <none>
I20260811 00:26:36.085055 8932 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:36.085281 8932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:36.085691 8932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:36.089511 8932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "5efc04a1afc64c108e06679159b03827"
format_stamp: "Formatted at 2026-08-11 00:26:36 on dist-test-slave-fd7l"
server_key: "223f6056013aa1aa7ace703832899d98"
server_key_iv: "ec31e1788560f5c704447950f8065400"
server_key_version: "encryptionkey@0"
I20260811 00:26:36.091315 8932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "5efc04a1afc64c108e06679159b03827"
format_stamp: "Formatted at 2026-08-11 00:26:36 on dist-test-slave-fd7l"
server_key: "223f6056013aa1aa7ace703832899d98"
server_key_iv: "ec31e1788560f5c704447950f8065400"
server_key_version: "encryptionkey@0"
I20260811 00:26:36.101125 8932 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.003s sys 0.008s
I20260811 00:26:36.106098 8948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:36.108023 8932 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260811 00:26:36.108304 8932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "5efc04a1afc64c108e06679159b03827"
format_stamp: "Formatted at 2026-08-11 00:26:36 on dist-test-slave-fd7l"
server_key: "223f6056013aa1aa7ace703832899d98"
server_key_iv: "ec31e1788560f5c704447950f8065400"
server_key_version: "encryptionkey@0"
I20260811 00:26:36.108610 8932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:36.129727 8932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:36.134876 8932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:36.135243 8932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:36.160282 8932 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:34903
I20260811 00:26:36.160301 9006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:34903 every 8 connection(s)
I20260811 00:26:36.161849 8932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:26:36.166033 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 8932
I20260811 00:26:36.166280 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:26:36.166980 2528 external_mini_cluster.cc:1428] Setting key 08154a7c2b108b8050e45a1218a3b7b2
I20260811 00:26:36.167847 9007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:36.186321 9007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827: Bootstrap starting.
Aug 11 00:26:36 dist-test-slave-fd7l krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:36.191036 9007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:36.192674 9007 log.cc:824] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:36.196363 9007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827: No bootstrap required, opened a new log
I20260811 00:26:36.199985 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:36.169634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49664 (local address 127.2.120.62:34903)
0811 00:26:36.171247 (+ 1613us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:36.171259 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:36.171304 (+ 45us) server_negotiation.cc:407] Connection header received
0811 00:26:36.175059 (+ 3755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:36.175096 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:36.175593 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:36.176132 (+ 539us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:36.178003 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:36.179365 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:36.181227 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:36.182068 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:36.182245 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:36.189002 (+ 6757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:36.189027 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:36.189057 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:36.189135 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:36.194040 (+ 4905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:36.195457 (+ 1417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:36.195467 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:36.195477 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:36.195736 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:36.196318 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:36.196352 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:36.196356 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:36.197214 (+ 858us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:36.197638 (+ 424us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:36.198257 (+ 619us) server_negotiation.cc:299] Negotiation successful
0811 00:26:36.198791 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1396,"thread_start_us":886,"threads_started":1}
I20260811 00:26:36.206234 9007 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5efc04a1afc64c108e06679159b03827" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 34903 } }
I20260811 00:26:36.206753 9007 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:36.206892 9007 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5efc04a1afc64c108e06679159b03827, State: Initialized, Role: FOLLOWER
I20260811 00:26:36.207697 9007 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5efc04a1afc64c108e06679159b03827" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 34903 } }
I20260811 00:26:36.207978 9007 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:26:36.208158 9007 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:26:36.208388 9007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:36.211514 9007 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5efc04a1afc64c108e06679159b03827" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 34903 } }
I20260811 00:26:36.212128 9007 leader_election.cc:302] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5efc04a1afc64c108e06679159b03827; no voters:
I20260811 00:26:36.213014 9007 leader_election.cc:288] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:26:36.213753 9012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:36.215344 9012 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827 [term 1 LEADER]: Becoming Leader. State: Replica: 5efc04a1afc64c108e06679159b03827, State: Running, Role: LEADER
I20260811 00:26:36.215997 9012 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5efc04a1afc64c108e06679159b03827" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 34903 } }
I20260811 00:26:36.216153 9007 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:26:36.219367 9013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5efc04a1afc64c108e06679159b03827" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5efc04a1afc64c108e06679159b03827" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 34903 } } }
I20260811 00:26:36.219800 9013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827 [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:36.220242 9014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5efc04a1afc64c108e06679159b03827. Latest consensus state: current_term: 1 leader_uuid: "5efc04a1afc64c108e06679159b03827" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5efc04a1afc64c108e06679159b03827" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 34903 } } }
I20260811 00:26:36.220650 9014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827 [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:36.223951 9021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:26:36.229935 9021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:26:36.242365 9021 catalog_manager.cc:1347] Generated new cluster ID: e6b85ab667f2422aa328f97ad41f9dd5
I20260811 00:26:36.242580 9021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:26:36.269387 9021 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:26:36.271512 9021 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:26:36.281457 9021 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5efc04a1afc64c108e06679159b03827: Generated new TSK 0
I20260811 00:26:36.282541 9021 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
I20260811 00:26:36.372310 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:34903
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:46281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:36.773612 9034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:36.773967 9034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:36.774130 9034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:36.783622 9034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:36.783854 9034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:36.784112 9034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:26:36.796813 9034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:46281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:34903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:36.798354 9034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:36.799851 9034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:36.815294 9039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:36.815335 9040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:36.815371 9042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:36.817323 9034 server_base.cc:1034] running on GCE node
I20260811 00:26:36.818166 9034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:36.819309 9034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:36.820647 9034 hybrid_clock.cc:648] HybridClock initialized: now 1786407996820561 us; error 74 us; skew 500 ppm
I20260811 00:26:36.821012 9034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:36 dist-test-slave-fd7l krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407996, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:36.825138 9034 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:26:36.827136 9034 webserver.cc:460] Webserver started at http://127.2.120.1:40047/ using document root <none> and password file <none>
I20260811 00:26:36.828229 9034 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:36.828373 9034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:36.828796 9034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:36.832257 9034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "ba41b66b9cd84bcf80f3f8ebde244ec1"
format_stamp: "Formatted at 2026-08-11 00:26:36 on dist-test-slave-fd7l"
server_key: "fd01779d1362ecc5d77427e11e3e0a23"
server_key_iv: "6318b9754f2b66ba54ad12b2104d8a13"
server_key_version: "encryptionkey@0"
I20260811 00:26:36.833324 9034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "ba41b66b9cd84bcf80f3f8ebde244ec1"
format_stamp: "Formatted at 2026-08-11 00:26:36 on dist-test-slave-fd7l"
server_key: "fd01779d1362ecc5d77427e11e3e0a23"
server_key_iv: "6318b9754f2b66ba54ad12b2104d8a13"
server_key_version: "encryptionkey@0"
I20260811 00:26:36.839952 9034 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 00:26:36.845017 9049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:36.847507 9034 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 00:26:36.847841 9034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "ba41b66b9cd84bcf80f3f8ebde244ec1"
format_stamp: "Formatted at 2026-08-11 00:26:36 on dist-test-slave-fd7l"
server_key: "fd01779d1362ecc5d77427e11e3e0a23"
server_key_iv: "6318b9754f2b66ba54ad12b2104d8a13"
server_key_version: "encryptionkey@0"
I20260811 00:26:36.848177 9034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:36.875267 9034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:36.881958 9034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:36.882396 9034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:36.883767 9034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:36.885878 9034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:36.886003 9034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:36.886176 9034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:36.886333 9034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:36.930094 9034 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:42785
I20260811 00:26:36.930131 9175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:42785 every 8 connection(s)
I20260811 00:26:36.931710 9034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:26:36.935436 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 9034
I20260811 00:26:36.935761 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:26:36.936533 2528 external_mini_cluster.cc:1428] Setting key d72b5db73948c6effd5e0dcb34142009
Aug 11 00:26:36 dist-test-slave-fd7l krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407996, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:36.965821 9178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:36.935291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:45621 (local address 127.2.120.62:34903)
0811 00:26:36.935923 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:36.935931 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:36.937378 (+ 1447us) server_negotiation.cc:407] Connection header received
0811 00:26:36.940990 (+ 3612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:36.940996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:36.941097 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:36.941227 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:36.944842 (+ 3615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:36.945683 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:36.949019 (+ 3336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:36.949595 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:36.949787 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:36.954854 (+ 5067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:36.954864 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:36.954871 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:36.954932 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:36.960028 (+ 5096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:36.961555 (+ 1527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:36.961568 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:36.961574 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:36.962003 (+ 429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:36.962984 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:36.962994 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:36.963001 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:36.963514 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:36.963707 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:36.964940 (+ 1233us) server_negotiation.cc:299] Negotiation successful
0811 00:26:36.965272 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":327,"threads_started":1}
I20260811 00:26:36.966825 9179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:36.935944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:34903 (local address 127.2.120.1:45621)
0811 00:26:36.937095 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:36.937169 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:36.940208 (+ 3039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:36.941769 (+ 1561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:36.941893 (+ 124us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:36.943156 (+ 1263us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:36.944293 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:36.944346 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:36.945970 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:36.945979 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:36.948745 (+ 2766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:36.948776 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:36.949896 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:36.949902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:36.950142 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:36.951007 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:36.951035 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:36.954387 (+ 3352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:36.960619 (+ 6232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:36.960631 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:36.960651 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:36.961271 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:36.962508 (+ 1237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:36.962512 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:36.962515 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:36.962792 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:36.963726 (+ 934us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:36.963734 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:36.964292 (+ 558us) client_negotiation.cc:769] Sending connection context
0811 00:26:36.964727 (+ 435us) client_negotiation.cc:240] Negotiation successful
0811 00:26:36.965253 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":451,"threads_started":1}
WARNING: no policy specified for kudu/127.2.120.2@KRBTEST.COM; defaulting to no policy
I20260811 00:26:36.969760 9176 heartbeater.cc:347] Connected to a master server at 127.2.120.62:34903
I20260811 00:26:36.970340 9176 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:36.971292 9176 heartbeater.cc:511] Master 127.2.120.62:34903 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.2@KRBTEST.COM" created.
I20260811 00:26:36.974933 8965 ts_manager.cc:194] Registered new tserver with Master: ba41b66b9cd84bcf80f3f8ebde244ec1 (127.2.120.1:42785)
I20260811 00:26:36.978001 8965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:45621
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
I20260811 00:26:37.010295 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:34903
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:46281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:37.399827 9182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:37.400238 9182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:37.400408 9182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:37.408245 9182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:37.408452 9182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:37.408663 9182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:26:37.419602 9182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:46281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:34903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:37.421140 9182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:37.422569 9182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:37.437268 9187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:37.437291 9190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:37.437629 9188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:37.438917 9182 server_base.cc:1034] running on GCE node
I20260811 00:26:37.439793 9182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:37.441280 9182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:37.442812 9182 hybrid_clock.cc:648] HybridClock initialized: now 1786407997442744 us; error 72 us; skew 500 ppm
I20260811 00:26:37.443403 9182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:37 dist-test-slave-fd7l krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407997, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:37.448156 9182 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:26:37.450497 9182 webserver.cc:460] Webserver started at http://127.2.120.2:36141/ using document root <none> and password file <none>
I20260811 00:26:37.451552 9182 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:37.451684 9182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:37.452021 9182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:37.456180 9182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "31e551e02a4d4a1780a0fea169767740"
format_stamp: "Formatted at 2026-08-11 00:26:37 on dist-test-slave-fd7l"
server_key: "3878222067fbdbc8887c69206b0fc147"
server_key_iv: "4e219873f5a44e17b08665ac04b732c1"
server_key_version: "encryptionkey@0"
I20260811 00:26:37.457260 9182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "31e551e02a4d4a1780a0fea169767740"
format_stamp: "Formatted at 2026-08-11 00:26:37 on dist-test-slave-fd7l"
server_key: "3878222067fbdbc8887c69206b0fc147"
server_key_iv: "4e219873f5a44e17b08665ac04b732c1"
server_key_version: "encryptionkey@0"
I20260811 00:26:37.465867 9182 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260811 00:26:37.471627 9198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:37.473570 9182 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 00:26:37.473817 9182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "31e551e02a4d4a1780a0fea169767740"
format_stamp: "Formatted at 2026-08-11 00:26:37 on dist-test-slave-fd7l"
server_key: "3878222067fbdbc8887c69206b0fc147"
server_key_iv: "4e219873f5a44e17b08665ac04b732c1"
server_key_version: "encryptionkey@0"
I20260811 00:26:37.474084 9182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:37.495597 9182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:37.501235 9182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:37.501628 9182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:37.503131 9182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:37.505543 9182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:37.505672 9182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:37.505858 9182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:37.505923 9182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:37.549187 9182 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:41889
I20260811 00:26:37.549245 9321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:41889 every 8 connection(s)
I20260811 00:26:37.550978 9182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:26:37.556320 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 9182
I20260811 00:26:37.556656 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:26:37.557338 2528 external_mini_cluster.cc:1428] Setting key 1252080a4dd1f1e2a256430a4125eb6d
Aug 11 00:26:37 dist-test-slave-fd7l krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407997, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:37.577145 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:37.554450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:45731 (local address 127.2.120.62:34903)
0811 00:26:37.555445 (+ 995us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:37.555453 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:37.557744 (+ 2291us) server_negotiation.cc:407] Connection header received
0811 00:26:37.560212 (+ 2468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:37.560218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:37.560323 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:37.560474 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:37.562477 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:37.563282 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:37.565243 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:37.565770 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:37.565942 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:37.570287 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:37.570293 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:37.570296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:37.570340 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:37.573410 (+ 3070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:37.574379 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:37.574384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:37.574386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:37.574520 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:37.575041 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:37.575046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:37.575048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:37.575360 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:37.575550 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:37.576405 (+ 855us) server_negotiation.cc:299] Negotiation successful
0811 00:26:37.576712 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":418,"threads_started":1}
I20260811 00:26:37.578337 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:37.555215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:34903 (local address 127.2.120.2:45731)
0811 00:26:37.557379 (+ 2164us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:37.557543 (+ 164us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:37.559849 (+ 2306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:37.560844 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:37.560854 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:37.561504 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:37.562221 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:37.562245 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:37.563500 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:37.563505 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:37.565036 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:37.565055 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:37.566067 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:37.566074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:37.566354 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:37.567205 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:37.567237 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:37.569906 (+ 2669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:37.573626 (+ 3720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:37.573638 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:37.573668 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:37.574208 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:37.574652 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:37.574657 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:37.574661 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:37.574895 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:37.575523 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:37.575532 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:37.575957 (+ 425us) client_negotiation.cc:769] Sending connection context
0811 00:26:37.576301 (+ 344us) client_negotiation.cc:240] Negotiation successful
0811 00:26:37.577032 (+ 731us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1836,"thread_start_us":1459,"threads_started":1}
I20260811 00:26:37.580408 9322 heartbeater.cc:347] Connected to a master server at 127.2.120.62:34903
I20260811 00:26:37.580857 9322 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:37.581785 9322 heartbeater.cc:511] Master 127.2.120.62:34903 requested a full tablet report, sending...
I20260811 00:26:37.584213 8965 ts_manager.cc:194] Registered new tserver with Master: 31e551e02a4d4a1780a0fea169767740 (127.2.120.2:41889)
I20260811 00:26:37.585621 8965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:45731
WARNING: no policy specified for kudu/127.2.120.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
I20260811 00:26:37.625382 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:34903
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:46281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:26:37.982656 9176 heartbeater.cc:503] Master 127.2.120.62:34903 was elected leader, sending a full tablet report...
W20260811 00:26:38.038343 9330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:38.038724 9330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:38.038918 9330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:38.046515 9330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:38.046733 9330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:38.046950 9330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:26:38.057826 9330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:46281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:34903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:38.059365 9330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:38.060840 9330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:38.075742 9337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:38.075742 9336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:38.075790 9339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:38.076776 9330 server_base.cc:1034] running on GCE node
I20260811 00:26:38.077658 9330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:38.079138 9330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:38.080431 9330 hybrid_clock.cc:648] HybridClock initialized: now 1786407998080373 us; error 62 us; skew 500 ppm
I20260811 00:26:38.080914 9330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:38 dist-test-slave-fd7l krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407998, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:38.084563 9330 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:26:38.086249 9330 webserver.cc:460] Webserver started at http://127.2.120.3:44403/ using document root <none> and password file <none>
I20260811 00:26:38.087096 9330 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:38.087216 9330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:38.087508 9330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:38.091042 9330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "438395d2c2574f8a880ff6a81229b16e"
format_stamp: "Formatted at 2026-08-11 00:26:38 on dist-test-slave-fd7l"
server_key: "4835fb3e95a01965b66ae4cca787ee24"
server_key_iv: "6914b44a6144c4bf10ad0b36beca9520"
server_key_version: "encryptionkey@0"
I20260811 00:26:38.091938 9330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "438395d2c2574f8a880ff6a81229b16e"
format_stamp: "Formatted at 2026-08-11 00:26:38 on dist-test-slave-fd7l"
server_key: "4835fb3e95a01965b66ae4cca787ee24"
server_key_iv: "6914b44a6144c4bf10ad0b36beca9520"
server_key_version: "encryptionkey@0"
I20260811 00:26:38.098232 9330 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.007s
I20260811 00:26:38.102269 9346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:38.103900 9330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:26:38.104092 9330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "438395d2c2574f8a880ff6a81229b16e"
format_stamp: "Formatted at 2026-08-11 00:26:38 on dist-test-slave-fd7l"
server_key: "4835fb3e95a01965b66ae4cca787ee24"
server_key_iv: "6914b44a6144c4bf10ad0b36beca9520"
server_key_version: "encryptionkey@0"
I20260811 00:26:38.104331 9330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:38.125304 9330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:38.130156 9330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:38.130491 9330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:38.131600 9330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:38.133195 9330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:38.133275 9330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:38.133453 9330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:38.133512 9330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:38.175006 9330 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:45541
I20260811 00:26:38.175041 9471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:45541 every 8 connection(s)
I20260811 00:26:38.176587 9330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:26:38.185690 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 9330
I20260811 00:26:38.185896 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:26:38.186383 2528 external_mini_cluster.cc:1428] Setting key 621fd114bf8a334f9c40cee68dadc40e
Aug 11 00:26:38 dist-test-slave-fd7l krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407998, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:26:38.201282 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:38.180650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:41855 (local address 127.2.120.62:34903)
0811 00:26:38.181578 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:38.181584 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:38.182792 (+ 1208us) server_negotiation.cc:407] Connection header received
0811 00:26:38.185078 (+ 2286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:38.185085 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:38.185189 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:38.185336 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:38.187279 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:38.188249 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:38.190015 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:38.190453 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:38.190613 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:38.194618 (+ 4005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:38.194624 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:38.194628 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:38.194667 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:38.197722 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:38.198571 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:38.198579 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:38.198592 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:38.198726 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:38.199209 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:38.199214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:38.199216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:38.199514 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:38.199692 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:38.200563 (+ 871us) server_negotiation.cc:299] Negotiation successful
0811 00:26:38.200842 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":348,"threads_started":1}
I20260811 00:26:38.202423 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:38.181357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:34903 (local address 127.2.120.3:41855)
0811 00:26:38.182591 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:38.182648 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:38.184666 (+ 2018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:38.185624 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:38.185634 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:38.186344 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:38.187074 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:38.187095 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:38.188453 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:38.188457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:38.189849 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:38.189864 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:38.190773 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:38.190784 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:38.191041 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:38.191777 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:38.191807 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:38.194311 (+ 2504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:38.197919 (+ 3608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:38.197929 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:38.197946 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:38.198426 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:38.198851 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:38.198857 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:38.198866 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:38.199063 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:38.199655 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:38.199665 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:38.200080 (+ 415us) client_negotiation.cc:769] Sending connection context
0811 00:26:38.200425 (+ 345us) client_negotiation.cc:240] Negotiation successful
0811 00:26:38.201053 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"thread_start_us":525,"threads_started":1}
I20260811 00:26:38.204128 9472 heartbeater.cc:347] Connected to a master server at 127.2.120.62:34903
I20260811 00:26:38.204566 9472 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:38.205482 9472 heartbeater.cc:511] Master 127.2.120.62:34903 requested a full tablet report, sending...
I20260811 00:26:38.207674 8965 ts_manager.cc:194] Registered new tserver with Master: 438395d2c2574f8a880ff6a81229b16e (127.2.120.3:45541)
I20260811 00:26:38.208815 8965 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:41855
I20260811 00:26:38.215881 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/test-admin.keytab.
Aug 11 00:26:38 dist-test-slave-fd7l krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:38.250907 2528 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 00:26:38.257658 9481 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:26:38.254588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60066 (local address 127.2.120.1:42785)
0811 00:26:38.255373 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:38.255381 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:38.255415 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:26:38.255484 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:38.255491 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:38.255731 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:38.255894 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:38.257144 (+ 1250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60066: BlockingRecv error: recv got EOF from 127.0.0.1:60066 (error 108)
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":465,"threads_started":1}
W20260811 00:26:38.257901 2528 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/krb5cc' of type FILE
Aug 11 00:26:38 dist-test-slave-fd7l krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:38.260636 2528 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 00:26:38 dist-test-slave-fd7l krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:38.262403 2528 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 00:26:38 dist-test-slave-fd7l krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:26:38.282306 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:38.264570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60076 (local address 127.2.120.1:42785)
0811 00:26:38.264860 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:38.264868 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:38.265450 (+ 582us) server_negotiation.cc:407] Connection header received
0811 00:26:38.265661 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:38.265665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:38.265740 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:38.265844 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:38.267031 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:38.268235 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:38.270174 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:38.270794 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:38.270919 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:38.275140 (+ 4221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:38.275157 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:38.275174 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:38.275222 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:38.278340 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:38.279346 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:38.279354 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:38.279357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:38.279501 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:38.279964 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:38.279969 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:38.279971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:38.280423 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:38.280792 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:38.281606 (+ 814us) server_negotiation.cc:299] Negotiation successful
0811 00:26:38.281919 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
W20260811 00:26:38.289837 9481 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:26:38.287179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60080 (local address 127.2.120.1:42785)
0811 00:26:38.287457 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:38.287464 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:38.287714 (+ 250us) server_negotiation.cc:407] Connection header received
0811 00:26:38.287846 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:38.287850 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:38.287918 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:38.288006 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:38.289449 (+ 1443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60080: BlockingRecv error: recv got EOF from 127.0.0.1:60080 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
W20260811 00:26:38.290967 2528 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/krb5cc)
Aug 11 00:26:38 dist-test-slave-fd7l krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:38.292893 2528 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 00:26:38 dist-test-slave-fd7l krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:38.294663 2528 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 00:26:38 dist-test-slave-fd7l krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:26:38.312788 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:38.296595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60094 (local address 127.2.120.1:42785)
0811 00:26:38.296862 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:38.296873 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:38.296967 (+ 94us) server_negotiation.cc:407] Connection header received
0811 00:26:38.297140 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:38.297145 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:38.297212 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:38.297306 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:38.298435 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:38.299243 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:38.300699 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:38.301225 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:38.301344 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:38.306272 (+ 4928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:38.306280 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:38.306284 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:38.306344 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:38.309648 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:38.310599 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:38.310605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:38.310608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:38.310781 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:38.311190 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:38.311194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:38.311197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:38.311501 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:38.311715 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:38.312094 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 00:26:38.312372 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 00:26:38.318396 9481 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:26:38.316030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60106 (local address 127.2.120.1:42785)
0811 00:26:38.316289 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:38.316296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:38.316664 (+ 368us) server_negotiation.cc:407] Connection header received
0811 00:26:38.316854 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:38.316859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:38.316946 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:38.317105 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:38.318101 (+ 996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60106: BlockingRecv error: recv got EOF from 127.0.0.1:60106 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 00:26:38.318534 2528 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407939260739-2528-0/krb5kdc/krb5cc' of type FILE
Aug 11 00:26:38 dist-test-slave-fd7l krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:38.320281 2528 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 00:26:38 dist-test-slave-fd7l krb5kdc[8917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:38.321802 2528 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 00:26:38 dist-test-slave-fd7l krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:26:38.338515 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:38.323495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60116 (local address 127.2.120.1:42785)
0811 00:26:38.323734 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:38.323740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:38.324032 (+ 292us) server_negotiation.cc:407] Connection header received
0811 00:26:38.324162 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:38.324167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:38.324241 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:38.324334 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:38.325439 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:38.326206 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:38.327528 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:38.327981 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:38.328098 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:38.332230 (+ 4132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:38.332236 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:38.332239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:38.332280 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:38.335085 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:38.335971 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:38.335976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:38.335979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:38.336164 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:38.336623 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:38.336628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:38.336630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:38.336956 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:38.337457 (+ 501us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:38.337668 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 00:26:38.338146 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 00:26:38.340348 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 9034
I20260811 00:26:38.354034 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:38.341288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60130 (local address 127.2.120.1:42785)
0811 00:26:38.341535 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:38.341543 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:38.341850 (+ 307us) server_negotiation.cc:407] Connection header received
0811 00:26:38.342059 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:38.342063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:38.342142 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:38.342254 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:38.343280 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:38.343993 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:38.345413 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:38.345888 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:38.346045 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:38.348391 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:38.348397 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:38.348400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:38.348437 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:38.351037 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:38.351800 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:38.351807 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:38.351810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:38.351969 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:38.352380 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:38.352386 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:38.352389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:38.352760 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:38.353089 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:38.353414 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 00:26:38.353674 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:26:38.354761 9171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:38.443801 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 9034
I20260811 00:26:38.466490 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 9182
Aug 11 00:26:38 dist-test-slave-fd7l krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.2@KRBTEST.COM
I20260811 00:26:38.486219 9495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:38.467691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42452 (local address 127.2.120.2:41889)
0811 00:26:38.468253 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:38.468261 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:38.468290 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:26:38.468366 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:38.468373 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:38.468717 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:38.468878 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:38.470144 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:38.471437 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:38.473086 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:38.473753 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:38.473885 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:38.479179 (+ 5294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:38.479198 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:38.479217 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:38.479262 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:38.482858 (+ 3596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:38.483759 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:38.483766 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:38.483769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:38.483918 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:38.484424 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:38.484429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:38.484431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:38.484982 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:38.485269 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:38.485633 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 00:26:38.485870 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":273,"threads_started":1}
I20260811 00:26:38.487623 9317 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:38.573505 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 9182
I20260811 00:26:38.594327 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 9330
Aug 11 00:26:38 dist-test-slave-fd7l krb5kdc[8917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.3@KRBTEST.COM
I20260811 00:26:38.615738 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:38.595500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54668 (local address 127.2.120.3:45541)
0811 00:26:38.596199 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:38.596207 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:38.596238 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:26:38.596305 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:38.596312 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:38.596613 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:38.596800 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:38.597962 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:38.599040 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:38.600365 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:38.601186 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:38.601360 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:38.607637 (+ 6277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:38.607659 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:38.607678 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:38.607728 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:38.611718 (+ 3990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:38.612832 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:38.612842 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:38.612845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:38.613102 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:38.613644 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:38.613650 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:38.613654 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:38.614206 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:38.614593 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:38.614989 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 00:26:38.615333 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":379,"threads_started":1}
I20260811 00:26:38.617538 9467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:38.706317 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 9330
I20260811 00:26:38.729378 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 8932
I20260811 00:26:38.730611 9000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:38.838557 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 8932
2026-08-11T00:26:38Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4719 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:26:38 dist-test-slave-fd7l krb5kdc[9503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:26:38 dist-test-slave-fd7l krb5kdc[9503](info): set up 2 sockets
Aug 11 00:26:38 dist-test-slave-fd7l krb5kdc[9503](info): commencing operation
krb5kdc: starting...
W20260811 00:26:40.105281 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:26:40 dist-test-slave-fd7l krb5kdc[9503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:26:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:26:40Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.62.keytab.
Entry for principal oryx/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.62.keytab.
I20260811 00:26:40.393882 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:33143
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.62.keytab
--principal=oryx/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:45261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:33143
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:40.786316 9519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:40.786909 9519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:40.787117 9519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:40.796221 9519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:26:40.796375 9519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:40.796425 9519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:40.796526 9519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:26:40.796641 9519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:26:40.807430 9519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:45261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:33143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.62.keytab
--principal=oryx/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:33143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:40.808990 9519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:40.810374 9519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:40.824041 9526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:40.823951 9528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:40.824080 9524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:40.824976 9519 server_base.cc:1034] running on GCE node
I20260811 00:26:40.825837 9519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:40.827410 9519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:40.828646 9519 hybrid_clock.cc:648] HybridClock initialized: now 1786408000828588 us; error 43 us; skew 500 ppm
I20260811 00:26:40.829041 9519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:40 dist-test-slave-fd7l krb5kdc[9503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408000, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:40.832636 9519 init.cc:377] Logged in from keytab as oryx/127.2.120.62@KRBTEST.COM (short username oryx)
I20260811 00:26:40.834349 9519 webserver.cc:460] Webserver started at http://127.2.120.62:33247/ using document root <none> and password file <none>
I20260811 00:26:40.835091 9519 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:40.835228 9519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:40.835580 9519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:40.838810 9519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "97e4014ce0a74657ab9899f0ba4c2eee"
format_stamp: "Formatted at 2026-08-11 00:26:40 on dist-test-slave-fd7l"
server_key: "b2633f43099e8962f69c551d9c4a60c2"
server_key_iv: "670ffecb11c4018ce8e1e3bf073ebf9c"
server_key_version: "encryptionkey@0"
I20260811 00:26:40.839623 9519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "97e4014ce0a74657ab9899f0ba4c2eee"
format_stamp: "Formatted at 2026-08-11 00:26:40 on dist-test-slave-fd7l"
server_key: "b2633f43099e8962f69c551d9c4a60c2"
server_key_iv: "670ffecb11c4018ce8e1e3bf073ebf9c"
server_key_version: "encryptionkey@0"
I20260811 00:26:40.845168 9519 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 00:26:40.848807 9535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:40.850185 9519 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:26:40.850402 9519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "97e4014ce0a74657ab9899f0ba4c2eee"
format_stamp: "Formatted at 2026-08-11 00:26:40 on dist-test-slave-fd7l"
server_key: "b2633f43099e8962f69c551d9c4a60c2"
server_key_iv: "670ffecb11c4018ce8e1e3bf073ebf9c"
server_key_version: "encryptionkey@0"
I20260811 00:26:40.850638 9519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:40.870761 9519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:40.875794 9519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:40.876138 9519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:40.897864 9519 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:33143
I20260811 00:26:40.897923 9592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:33143 every 8 connection(s)
I20260811 00:26:40.899425 9519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:26:40.903568 9593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:40.904218 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 9519
I20260811 00:26:40.904469 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:26:40.905210 2528 external_mini_cluster.cc:1428] Setting key 9849156923b4a348dcb67f37b6604ae8
I20260811 00:26:40.920694 9593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee: Bootstrap starting.
Aug 11 00:26:40 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM
I20260811 00:26:40.924532 9593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:40.926136 9593 log.cc:824] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:40.929986 9593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee: No bootstrap required, opened a new log
I20260811 00:26:40.934425 9596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:40.907488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56522 (local address 127.2.120.62:33143)
0811 00:26:40.908355 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:40.908382 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:40.908438 (+ 56us) server_negotiation.cc:407] Connection header received
0811 00:26:40.910911 (+ 2473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:40.910951 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:40.911630 (+ 679us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:40.912187 (+ 557us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:40.914036 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:40.916548 (+ 2512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:40.918054 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:40.919253 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:40.919475 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:40.924435 (+ 4960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:40.924450 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:40.924474 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:40.924573 (+ 99us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:40.928679 (+ 4106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:40.929507 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:40.929516 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:40.929526 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:40.929760 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:40.930314 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:40.930323 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:40.930329 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:40.931463 (+ 1134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:40.931852 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:40.932314 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 00:26:40.932891 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":367,"threads_started":1}
I20260811 00:26:40.940244 9593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } }
I20260811 00:26:40.941129 9593 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:40.941476 9593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e4014ce0a74657ab9899f0ba4c2eee, State: Initialized, Role: FOLLOWER
I20260811 00:26:40.942239 9593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } }
I20260811 00:26:40.942597 9593 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:26:40.942723 9593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:26:40.942859 9593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:40.945776 9593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } }
I20260811 00:26:40.946348 9593 leader_election.cc:302] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97e4014ce0a74657ab9899f0ba4c2eee; no voters:
I20260811 00:26:40.947085 9593 leader_election.cc:288] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:26:40.947264 9598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:40.948107 9598 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [term 1 LEADER]: Becoming Leader. State: Replica: 97e4014ce0a74657ab9899f0ba4c2eee, State: Running, Role: LEADER
I20260811 00:26:40.948804 9598 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } }
I20260811 00:26:40.949636 9593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:26:40.952126 9599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } }
I20260811 00:26:40.952713 9599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:40.952099 9600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97e4014ce0a74657ab9899f0ba4c2eee. Latest consensus state: current_term: 1 leader_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } }
I20260811 00:26:40.952963 9600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:40.958801 9605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:26:40.964475 9605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:26:40.975769 9605 catalog_manager.cc:1347] Generated new cluster ID: be18ea85b5ef4dc29a31e146ebce0964
I20260811 00:26:40.975915 9605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:26:40.994268 9605 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:26:40.996423 9605 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:26:41.008651 9605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee: Generated new TSK 0
I20260811 00:26:41.009649 9605 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.1.keytab.
Entry for principal oryx/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.1.keytab.
I20260811 00:26:41.096747 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:33143
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.1.keytab
--principal=oryx/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:45261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:41.470350 9622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:41.470968 9622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:41.471197 9622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:41.482719 9622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:41.482888 9622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:41.483073 9622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:26:41.494501 9622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:45261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.1.keytab
--principal=oryx/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:33143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:41.495921 9622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:41.497390 9622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:41.511597 9628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:41.511775 9627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:41.511938 9630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:41.512840 9622 server_base.cc:1034] running on GCE node
I20260811 00:26:41.513566 9622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:41.514649 9622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:41.515947 9622 hybrid_clock.cc:648] HybridClock initialized: now 1786408001515857 us; error 67 us; skew 500 ppm
I20260811 00:26:41.516348 9622 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:41 dist-test-slave-fd7l krb5kdc[9503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408001, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:41.525568 9622 init.cc:377] Logged in from keytab as oryx/127.2.120.1@KRBTEST.COM (short username oryx)
I20260811 00:26:41.527463 9622 webserver.cc:460] Webserver started at http://127.2.120.1:36037/ using document root <none> and password file <none>
I20260811 00:26:41.528308 9622 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:41.528486 9622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:41.528877 9622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:41.532311 9622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "0dcaa36e9c34456da1814e7cc9e27ace"
format_stamp: "Formatted at 2026-08-11 00:26:41 on dist-test-slave-fd7l"
server_key: "a5b096ad16374b1ae13bd2104b39664b"
server_key_iv: "086c77158a1a6f590a028c2e3cfc30ac"
server_key_version: "encryptionkey@0"
I20260811 00:26:41.533329 9622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "0dcaa36e9c34456da1814e7cc9e27ace"
format_stamp: "Formatted at 2026-08-11 00:26:41 on dist-test-slave-fd7l"
server_key: "a5b096ad16374b1ae13bd2104b39664b"
server_key_iv: "086c77158a1a6f590a028c2e3cfc30ac"
server_key_version: "encryptionkey@0"
I20260811 00:26:41.539398 9622 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.005s
I20260811 00:26:41.543586 9638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:41.545351 9622 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 00:26:41.545586 9622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "0dcaa36e9c34456da1814e7cc9e27ace"
format_stamp: "Formatted at 2026-08-11 00:26:41 on dist-test-slave-fd7l"
server_key: "a5b096ad16374b1ae13bd2104b39664b"
server_key_iv: "086c77158a1a6f590a028c2e3cfc30ac"
server_key_version: "encryptionkey@0"
I20260811 00:26:41.545837 9622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:41.568710 9622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:41.574285 9622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:41.574887 9622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:41.576081 9622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:41.577958 9622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:41.578044 9622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:41.578137 9622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:41.578181 9622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:41.618482 9622 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:38089
I20260811 00:26:41.618542 9767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:38089 every 8 connection(s)
I20260811 00:26:41.619866 9622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:26:41.624814 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 9622
I20260811 00:26:41.625046 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:26:41.625720 2528 external_mini_cluster.cc:1428] Setting key 8f9abc873c1d6130cb11f83a61134c61
Aug 11 00:26:41 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408001, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM
I20260811 00:26:41.644255 9770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:41.622709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:55195 (local address 127.2.120.62:33143)
0811 00:26:41.623271 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:41.623277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:41.624268 (+ 991us) server_negotiation.cc:407] Connection header received
0811 00:26:41.626453 (+ 2185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:41.626461 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:41.626560 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:41.626684 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:41.628952 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:41.630258 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:41.632104 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:41.632920 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:41.633092 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:41.637491 (+ 4399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:41.637497 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:41.637500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:41.637538 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:41.640655 (+ 3117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:41.641549 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:41.641556 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:41.641560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:41.641712 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:41.642252 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:41.642257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:41.642259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:41.642559 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:41.642745 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:41.643572 (+ 827us) server_negotiation.cc:299] Negotiation successful
0811 00:26:41.643849 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":329,"threads_started":1}
I20260811 00:26:41.645501 9771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:41.623152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.1:55195)
0811 00:26:41.623967 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:41.624021 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:41.626147 (+ 2126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:41.627017 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:41.627031 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:41.627952 (+ 921us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:41.628744 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:41.628767 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:41.630484 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:41.630498 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:41.631921 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:41.631936 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:41.633372 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:41.633378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:41.633636 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:41.634616 (+ 980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:41.634644 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:41.637212 (+ 2568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:41.640832 (+ 3620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:41.640844 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:41.640865 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:41.641393 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:41.641848 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:41.641854 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:41.641858 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:41.642081 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:41.642725 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:41.642734 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:41.643135 (+ 401us) client_negotiation.cc:769] Sending connection context
0811 00:26:41.643440 (+ 305us) client_negotiation.cc:240] Negotiation successful
0811 00:26:41.644066 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":311,"threads_started":1}
I20260811 00:26:41.647572 9768 heartbeater.cc:347] Connected to a master server at 127.2.120.62:33143
I20260811 00:26:41.647938 9768 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:41.648847 9768 heartbeater.cc:511] Master 127.2.120.62:33143 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.2.120.2@KRBTEST.COM; defaulting to no policy
I20260811 00:26:41.651683 9554 ts_manager.cc:194] Registered new tserver with Master: 0dcaa36e9c34456da1814e7cc9e27ace (127.2.120.1:38089)
I20260811 00:26:41.653311 9554 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:55195
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.120.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.2.keytab.
Entry for principal oryx/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.2.keytab.
I20260811 00:26:41.690019 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:33143
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.2.keytab
--principal=oryx/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:45261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:42.052127 9774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:42.052454 9774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:42.052623 9774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:42.060307 9774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:42.060433 9774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:42.060626 9774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:26:42.070783 9774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:45261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.2.keytab
--principal=oryx/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:33143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:42.072494 9774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:42.073958 9774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:42.088313 9779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:42.088261 9782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:42.088222 9780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:42.089803 9774 server_base.cc:1034] running on GCE node
I20260811 00:26:42.090611 9774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:42.091964 9774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:42.093258 9774 hybrid_clock.cc:648] HybridClock initialized: now 1786408002093187 us; error 66 us; skew 500 ppm
I20260811 00:26:42.093672 9774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:42 dist-test-slave-fd7l krb5kdc[9503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408002, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:42.097460 9774 init.cc:377] Logged in from keytab as oryx/127.2.120.2@KRBTEST.COM (short username oryx)
I20260811 00:26:42.099346 9774 webserver.cc:460] Webserver started at http://127.2.120.2:39953/ using document root <none> and password file <none>
I20260811 00:26:42.100193 9774 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:42.100366 9774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:42.100786 9774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:42.104203 9774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "2150e97860df4cc7bbfbf9b18dc8f86d"
format_stamp: "Formatted at 2026-08-11 00:26:42 on dist-test-slave-fd7l"
server_key: "23f0f8bfe9d83a52b47b1a66cd570548"
server_key_iv: "34eaedd5d01390c0c563a35e559a9c32"
server_key_version: "encryptionkey@0"
I20260811 00:26:42.105360 9774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "2150e97860df4cc7bbfbf9b18dc8f86d"
format_stamp: "Formatted at 2026-08-11 00:26:42 on dist-test-slave-fd7l"
server_key: "23f0f8bfe9d83a52b47b1a66cd570548"
server_key_iv: "34eaedd5d01390c0c563a35e559a9c32"
server_key_version: "encryptionkey@0"
I20260811 00:26:42.111279 9774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 00:26:42.115186 9789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:42.117000 9774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:26:42.117220 9774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "2150e97860df4cc7bbfbf9b18dc8f86d"
format_stamp: "Formatted at 2026-08-11 00:26:42 on dist-test-slave-fd7l"
server_key: "23f0f8bfe9d83a52b47b1a66cd570548"
server_key_iv: "34eaedd5d01390c0c563a35e559a9c32"
server_key_version: "encryptionkey@0"
I20260811 00:26:42.117460 9774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:42.133731 9774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:42.138824 9774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:42.139174 9774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:42.140296 9774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:42.141978 9774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:42.142100 9774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:42.142298 9774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:42.142381 9774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:42.181955 9774 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:45043
I20260811 00:26:42.182009 9913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:45043 every 8 connection(s)
I20260811 00:26:42.183462 9774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:26:42.187582 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 9774
I20260811 00:26:42.187813 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:26:42.188475 2528 external_mini_cluster.cc:1428] Setting key 09dad295c3f210789e51304ce77d2f62
Aug 11 00:26:42 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408002, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM
I20260811 00:26:42.209270 9917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:42.187170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.2:35033)
0811 00:26:42.188186 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:42.188242 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:42.190647 (+ 2405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:42.191465 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:42.191475 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:42.192104 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:42.192878 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:42.192900 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:42.194122 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:42.194126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:42.195511 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:42.195526 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:42.196465 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:42.196474 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:42.196781 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:42.197575 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:42.197608 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:42.200238 (+ 2630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:42.203805 (+ 3567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:42.203816 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:42.203839 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:42.204603 (+ 764us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:42.205145 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:42.205153 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:42.205159 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:42.205509 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:42.206340 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:42.206352 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:42.206948 (+ 596us) client_negotiation.cc:769] Sending connection context
0811 00:26:42.207312 (+ 364us) client_negotiation.cc:240] Negotiation successful
0811 00:26:42.207868 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":462,"threads_started":1}
I20260811 00:26:42.209450 9916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:42.186532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:35033 (local address 127.2.120.62:33143)
0811 00:26:42.187092 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:42.187099 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:42.188391 (+ 1292us) server_negotiation.cc:407] Connection header received
0811 00:26:42.190947 (+ 2556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:42.190953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:42.191042 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:42.191193 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:42.193087 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:42.193927 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:42.195706 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:42.196196 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:42.196346 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:42.200560 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:42.200569 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:42.200581 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:42.200624 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:42.203603 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:42.204805 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:42.204811 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:42.204814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:42.204989 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:42.205714 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:42.205722 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:42.205726 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:42.206178 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:42.206383 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:42.208747 (+ 2364us) server_negotiation.cc:299] Negotiation successful
0811 00:26:42.209017 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":274,"threads_started":1}
I20260811 00:26:42.211097 9914 heartbeater.cc:347] Connected to a master server at 127.2.120.62:33143
I20260811 00:26:42.211472 9914 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:42.212291 9914 heartbeater.cc:511] Master 127.2.120.62:33143 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.2.120.3@KRBTEST.COM; defaulting to no policy
I20260811 00:26:42.213987 9554 ts_manager.cc:194] Registered new tserver with Master: 2150e97860df4cc7bbfbf9b18dc8f86d (127.2.120.2:45043)
I20260811 00:26:42.214865 9554 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:35033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.120.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.3.keytab.
Entry for principal oryx/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.3.keytab.
I20260811 00:26:42.243993 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:33143
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.3.keytab
--principal=oryx/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:45261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:42.628803 9920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:42.629117 9920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:42.629278 9920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:42.636926 9920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:42.637097 9920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:42.637302 9920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:26:42.647480 9920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:45261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.3.keytab
--principal=oryx/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:33143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:42.649034 9920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:42.650756 9920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 00:26:42.657037 9768 heartbeater.cc:503] Master 127.2.120.62:33143 was elected leader, sending a full tablet report...
W20260811 00:26:42.666013 9928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:42.666013 9925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:42.665999 9926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:42.667018 9920 server_base.cc:1034] running on GCE node
I20260811 00:26:42.667788 9920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:42.668973 9920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:42.670223 9920 hybrid_clock.cc:648] HybridClock initialized: now 1786408002670151 us; error 59 us; skew 500 ppm
I20260811 00:26:42.670658 9920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:42 dist-test-slave-fd7l krb5kdc[9503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408002, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:42.674222 9920 init.cc:377] Logged in from keytab as oryx/127.2.120.3@KRBTEST.COM (short username oryx)
I20260811 00:26:42.675925 9920 webserver.cc:460] Webserver started at http://127.2.120.3:38883/ using document root <none> and password file <none>
I20260811 00:26:42.676775 9920 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:42.676915 9920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:42.677246 9920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:42.680353 9920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "e0691687a81d4375ac1258533cdce5eb"
format_stamp: "Formatted at 2026-08-11 00:26:42 on dist-test-slave-fd7l"
server_key: "26c3fcc1af1c3cde9fcd7bef8fc9eb5c"
server_key_iv: "12b62ccd3bc24b0f97443a340f1b8d25"
server_key_version: "encryptionkey@0"
I20260811 00:26:42.681284 9920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "e0691687a81d4375ac1258533cdce5eb"
format_stamp: "Formatted at 2026-08-11 00:26:42 on dist-test-slave-fd7l"
server_key: "26c3fcc1af1c3cde9fcd7bef8fc9eb5c"
server_key_iv: "12b62ccd3bc24b0f97443a340f1b8d25"
server_key_version: "encryptionkey@0"
I20260811 00:26:42.686786 9920 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:26:42.690793 9935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:42.692333 9920 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:26:42.692521 9920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "e0691687a81d4375ac1258533cdce5eb"
format_stamp: "Formatted at 2026-08-11 00:26:42 on dist-test-slave-fd7l"
server_key: "26c3fcc1af1c3cde9fcd7bef8fc9eb5c"
server_key_iv: "12b62ccd3bc24b0f97443a340f1b8d25"
server_key_version: "encryptionkey@0"
I20260811 00:26:42.692744 9920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:42.707448 9920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:42.711521 9920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:42.711848 9920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:42.712865 9920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:42.714380 9920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:42.714464 9920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:42.714617 9920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:42.714672 9920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:42.750983 9920 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:43095
I20260811 00:26:42.751026 10062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:43095 every 8 connection(s)
I20260811 00:26:42.752327 9920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:26:42.753036 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 9920
I20260811 00:26:42.753244 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:26:42.753845 2528 external_mini_cluster.cc:1428] Setting key 0ce9d6eb853616f4b5e751c5a5e3c176
Aug 11 00:26:42 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408002, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM
I20260811 00:26:42.775117 10065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:42.755178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:40183 (local address 127.2.120.62:33143)
0811 00:26:42.757640 (+ 2462us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:42.757646 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:42.757668 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:26:42.759183 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:42.759188 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:42.759294 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:42.759427 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:42.761144 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:42.761925 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:42.763510 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:42.763951 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:42.764091 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:42.767943 (+ 3852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:42.767949 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:42.767953 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:42.767995 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:42.771658 (+ 3663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:42.772539 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:42.772547 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:42.772551 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:42.772689 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:42.773169 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:42.773174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:42.773178 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:42.773482 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:42.773652 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:42.774439 (+ 787us) server_negotiation.cc:299] Negotiation successful
0811 00:26:42.774701 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2337,"thread_start_us":2185,"threads_started":1}
I20260811 00:26:42.776281 10066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:42.755847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.3:40183)
0811 00:26:42.756791 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:42.756849 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:42.758887 (+ 2038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:42.759691 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:42.759701 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:42.760255 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:42.760957 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:42.760975 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:42.762098 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:42.762103 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:42.763358 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:42.763371 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:42.764200 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:42.764207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:42.764442 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:42.765255 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:42.765281 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:42.767661 (+ 2380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:42.771829 (+ 4168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:42.771840 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:42.771868 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:42.772357 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:42.772810 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:42.772815 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:42.772819 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:42.773037 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:42.773618 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:42.773627 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:42.774036 (+ 409us) client_negotiation.cc:769] Sending connection context
0811 00:26:42.774321 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 00:26:42.774898 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":379,"threads_started":1}
I20260811 00:26:42.777869 10063 heartbeater.cc:347] Connected to a master server at 127.2.120.62:33143
I20260811 00:26:42.778254 10063 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:42.779067 10063 heartbeater.cc:511] Master 127.2.120.62:33143 requested a full tablet report, sending...
I20260811 00:26:42.780799 9554 ts_manager.cc:194] Registered new tserver with Master: e0691687a81d4375ac1258533cdce5eb (127.2.120.3:43095)
I20260811 00:26:42.781762 9554 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:40183
I20260811 00:26:42.783663 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.2.120.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.120.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.120.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.61.keytab.
Entry for principal oryx/127.2.120.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.61.keytab.
I20260811 00:26:42.841305 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.61:46301
--webserver_interface=127.2.120.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.61.keytab
--principal=oryx/127.2.120.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:45261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:33143,127.2.120.61:46301
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:26:43.217777 9914 heartbeater.cc:503] Master 127.2.120.62:33143 was elected leader, sending a full tablet report...
W20260811 00:26:43.220158 10069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:43.220517 10069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:43.220781 10069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:43.228663 10069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:26:43.228809 10069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:43.228914 10069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:43.229030 10069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:26:43.229116 10069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:26:43.239408 10069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:45261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:33143,127.2.120.61:46301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.61.keytab
--principal=oryx/127.2.120.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.61:46301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.120.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
W20260811 00:26:43.239877 10069 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.120.62:33143,127.2.120.61:46301'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 00:26:43.241109 10069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:43.242486 10069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:43.254940 10074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:43.254915 10075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:43.256363 10077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:43.256853 10069 server_base.cc:1034] running on GCE node
I20260811 00:26:43.257931 10069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:43.259887 10069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:43.261175 10069 hybrid_clock.cc:648] HybridClock initialized: now 1786408003261096 us; error 61 us; skew 500 ppm
I20260811 00:26:43.261678 10069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:43 dist-test-slave-fd7l krb5kdc[9503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408003, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:43.266264 10069 init.cc:377] Logged in from keytab as oryx/127.2.120.61@KRBTEST.COM (short username oryx)
I20260811 00:26:43.268013 10069 webserver.cc:460] Webserver started at http://127.2.120.61:45223/ using document root <none> and password file <none>
I20260811 00:26:43.268875 10069 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:43.269032 10069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:43.269390 10069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:43.272688 10069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/data/instance:
uuid: "5f58553db3fb46c49fa58a3ea3838cba"
format_stamp: "Formatted at 2026-08-11 00:26:43 on dist-test-slave-fd7l"
server_key: "27e367f878d6d12073e46b3e89bd05ee"
server_key_iv: "22c5d64ae0832a72f40973080d85e3ae"
server_key_version: "encryptionkey@0"
I20260811 00:26:43.273586 10069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/wal/instance:
uuid: "5f58553db3fb46c49fa58a3ea3838cba"
format_stamp: "Formatted at 2026-08-11 00:26:43 on dist-test-slave-fd7l"
server_key: "27e367f878d6d12073e46b3e89bd05ee"
server_key_iv: "22c5d64ae0832a72f40973080d85e3ae"
server_key_version: "encryptionkey@0"
I20260811 00:26:43.279785 10069 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:26:43.283790 10084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:43.285526 10069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 00:26:43.285749 10069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/wal
uuid: "5f58553db3fb46c49fa58a3ea3838cba"
format_stamp: "Formatted at 2026-08-11 00:26:43 on dist-test-slave-fd7l"
server_key: "27e367f878d6d12073e46b3e89bd05ee"
server_key_iv: "22c5d64ae0832a72f40973080d85e3ae"
server_key_version: "encryptionkey@0"
I20260811 00:26:43.286012 10069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:43.302564 10069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:43.307765 10069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:43.308096 10069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:43.330446 10069 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.61:46301
I20260811 00:26:43.330470 10141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.61:46301 every 8 connection(s)
I20260811 00:26:43.332158 10069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/data/info.pb
I20260811 00:26:43.336723 10142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:43.342031 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 10069
I20260811 00:26:43.344282 10142 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:26:43 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408003, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.61@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM
Aug 11 00:26:43 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.120.61@KRBTEST.COM
I20260811 00:26:43.366540 10146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:43.347318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43718 (local address 127.2.120.62:33143)
0811 00:26:43.347933 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:43.347941 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:43.348554 (+ 613us) server_negotiation.cc:407] Connection header received
0811 00:26:43.349139 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:43.349145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:43.349224 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:43.349440 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:43.350951 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.351851 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.353581 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.354118 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.354262 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:43.359452 (+ 5190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:43.359459 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:43.359462 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:43.359510 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:43.362832 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:43.363758 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:43.363763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:43.363766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:43.363984 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:43.364491 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:43.364505 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:43.364553 (+ 48us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:43.365001 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:43.365214 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:43.365857 (+ 643us) server_negotiation.cc:299] Negotiation successful
0811 00:26:43.366148 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":320,"threads_started":1}
I20260811 00:26:43.367784 10145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:43.344671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48964 (local address 127.2.120.61:46301)
0811 00:26:43.345625 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:43.345639 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:43.345687 (+ 48us) server_negotiation.cc:407] Connection header received
0811 00:26:43.348383 (+ 2696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:43.348418 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:43.348870 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:43.349197 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:43.350990 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.352399 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.353750 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.354637 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.354793 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:43.359914 (+ 5121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:43.359925 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:43.359942 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:43.359984 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:43.363174 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:43.363997 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:43.364010 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:43.364018 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:43.364198 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:43.364723 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:43.364732 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:43.364735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:43.365362 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:43.365761 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:43.366100 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 00:26:43.366551 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":187,"server-negotiator.queue_time_us":703,"spinlock_wait_cycles":21760,"thread_start_us":455,"threads_started":1}
I20260811 00:26:43.367835 10147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:43.347314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:43718)
0811 00:26:43.348310 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:43.348349 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:43.348822 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:43.349520 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:43.349529 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:43.350337 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:43.350747 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:43.350766 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.352017 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.352023 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:43.353411 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:43.353433 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.354397 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.354406 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:43.354679 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:43.355587 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:43.355635 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:43.359151 (+ 3516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:43.363033 (+ 3882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:43.363049 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:43.363065 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:43.363591 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:43.364134 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:43.364139 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:43.364143 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:43.364349 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:43.365168 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:43.365177 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:43.365431 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 00:26:43.365747 (+ 316us) client_negotiation.cc:240] Negotiation successful
0811 00:26:43.366377 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"mutex_wait_us":10,"spinlock_wait_cycles":15104,"thread_start_us":597,"threads_started":1}
I20260811 00:26:43.370265 10142 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:26:43 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408003, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.61@KRBTEST.COM for oryx/127.2.120.61@KRBTEST.COM
I20260811 00:26:43.383482 10147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:43.371309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:46301 (local address 127.0.0.1:48976)
0811 00:26:43.371495 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:43.371517 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:43.371636 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:43.371903 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:43.371906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:43.372269 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:43.372404 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:43.372412 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.373427 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.373431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:43.374320 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:43.374331 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.375035 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.375041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:43.375207 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:43.375798 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:43.375816 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:43.377973 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:43.381084 (+ 3111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:43.381089 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:43.381092 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:43.381491 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:43.381935 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:43.381939 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:43.381941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:43.382097 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:43.382705 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:43.382708 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:43.382846 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 00:26:43.382968 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 00:26:43.383235 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 00:26:43.383853 10145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:43.371336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48976 (local address 127.2.120.61:46301)
0811 00:26:43.371512 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:43.371516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:43.371584 (+ 68us) server_negotiation.cc:407] Connection header received
0811 00:26:43.371729 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:43.371733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:43.371788 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:43.371868 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:43.372594 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.373286 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.374448 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.374871 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.374970 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:43.378175 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:43.378180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:43.378184 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:43.378212 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:43.380916 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:43.381640 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:43.381644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:43.381647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:43.381790 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:43.382247 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:43.382251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:43.382253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:43.382582 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:43.382721 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:43.383063 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 00:26:43.383456 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":136,"server-negotiator.queue_time_us":65}
I20260811 00:26:43.389067 10142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: Bootstrap starting.
I20260811 00:26:43.392867 10142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:43.394016 10142 log.cc:824] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:43.397339 10142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: No bootstrap required, opened a new log
I20260811 00:26:43.404876 10142 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } }
I20260811 00:26:43.405280 10142 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:43.405355 10142 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f58553db3fb46c49fa58a3ea3838cba, State: Initialized, Role: FOLLOWER
I20260811 00:26:43.405949 10142 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } }
I20260811 00:26:43.406944 10149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } } }
I20260811 00:26:43.407305 10149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:26:43.408601 10142 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:26:43.412374 10149 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:26:43.412776 10149 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } }
I20260811 00:26:43.414788 10149 leader_election.cc:288] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97e4014ce0a74657ab9899f0ba4c2eee (127.2.120.62:33143)
I20260811 00:26:43.415961 9565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f58553db3fb46c49fa58a3ea3838cba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" is_pre_election: true
I20260811 00:26:43.417452 10085 leader_election.cc:302] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 5f58553db3fb46c49fa58a3ea3838cba; no voters: 97e4014ce0a74657ab9899f0ba4c2eee
I20260811 00:26:43.418555 10149 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260811 00:26:43.419932 10161 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:26:43.420084 10161 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:26:43.423496 10069 master_runner.cc:428] Detected that this master 5f58553db3fb46c49fa58a3ea3838cba is joining an existing cluster
I20260811 00:26:43.423594 10069 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.120.61:46301
I20260811 00:26:43.456390 10146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:43.432709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43722 (local address 127.2.120.62:33143)
0811 00:26:43.432944 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:43.432953 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:43.438247 (+ 5294us) server_negotiation.cc:407] Connection header received
0811 00:26:43.438591 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:43.438597 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:43.438713 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:43.438835 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:43.440572 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.441824 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.444244 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.445020 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.445144 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:43.448142 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:43.448152 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:43.448156 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:43.448209 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:43.452447 (+ 4238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:43.453432 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:43.453441 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:43.453448 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:43.453620 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:43.454183 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:43.454192 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:43.454198 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:43.454619 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:43.454757 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:43.455551 (+ 794us) server_negotiation.cc:299] Negotiation successful
0811 00:26:43.455913 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:26:43.458158 10169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:43.432651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:43722)
0811 00:26:43.438039 (+ 5388us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:43.438076 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:43.438330 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:43.439054 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:43.439065 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:43.439826 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:43.440290 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:43.440306 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.442212 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.442219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:43.443983 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:43.444020 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.445509 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.445517 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:43.445718 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:43.446502 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:43.446527 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:43.447835 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:43.452659 (+ 4824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:43.452668 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:43.452675 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:43.453271 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:43.453802 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:43.453808 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:43.453811 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:43.454031 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:43.454868 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:43.454877 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:43.455159 (+ 282us) client_negotiation.cc:769] Sending connection context
0811 00:26:43.457090 (+ 1931us) client_negotiation.cc:240] Negotiation successful
0811 00:26:43.457503 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5089,"thread_start_us":15152,"threads_started":1}
WARNING: no policy specified for oryx/127.2.120.60@KRBTEST.COM; defaulting to no policy
I20260811 00:26:43.465160 10145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:43.433823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48992 (local address 127.2.120.61:46301)
0811 00:26:43.434052 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:43.434061 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:43.447812 (+ 13751us) server_negotiation.cc:407] Connection header received
0811 00:26:43.448328 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:43.448336 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:43.448414 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:43.448491 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:43.450120 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.451623 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.453661 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.454436 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.454536 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:43.457406 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:43.457413 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:43.457418 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:43.457459 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:43.461252 (+ 3793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:43.462255 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:43.462262 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:43.462265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:43.462807 (+ 542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:43.463322 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:43.463328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:43.463331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:43.463760 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:43.463902 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:43.464380 (+ 478us) server_negotiation.cc:299] Negotiation successful
0811 00:26:43.464721 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:26:43.466251 10168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:43.433291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:46301 (local address 127.0.0.1:48992)
0811 00:26:43.447686 (+ 14395us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:43.447719 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:43.448180 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:43.448740 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:43.448748 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:43.449566 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:43.449955 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:43.449971 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.451844 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.451850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:43.453497 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:43.453512 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.454945 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.454952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:43.455168 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:43.455944 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:43.455967 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:43.457149 (+ 1182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:43.461394 (+ 4245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:43.461402 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:43.461406 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:43.462075 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:43.462952 (+ 877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:43.462957 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:43.462961 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:43.463191 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:43.463969 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:43.463974 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:43.464122 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 00:26:43.465422 (+ 1300us) client_negotiation.cc:240] Negotiation successful
0811 00:26:43.465767 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14164,"thread_start_us":4200,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.120.60@KRBTEST.COM" created.
Aug 11 00:26:43 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408000, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.62@KRBTEST.COM for oryx/127.2.120.61@KRBTEST.COM
I20260811 00:26:43.496064 10145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:43.466691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48996 (local address 127.2.120.61:46301)
0811 00:26:43.468102 (+ 1411us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:43.468111 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:43.476325 (+ 8214us) server_negotiation.cc:407] Connection header received
0811 00:26:43.476811 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:43.476818 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:43.476949 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:43.477041 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:43.478457 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.479604 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.481930 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.482684 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.482838 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:43.487647 (+ 4809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:43.487657 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:43.487665 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:43.487721 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:43.492363 (+ 4642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:43.493499 (+ 1136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:43.493507 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:43.493513 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:43.493703 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:43.494205 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:43.494213 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:43.494217 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:43.494639 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:43.494813 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:43.495393 (+ 580us) server_negotiation.cc:299] Negotiation successful
0811 00:26:43.495614 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1286}
I20260811 00:26:43.496215 10171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:43.466263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:46301 (local address 127.0.0.1:48996)
0811 00:26:43.476113 (+ 9850us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:43.476171 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:43.476452 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:43.477340 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:43.477346 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:43.478016 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:43.478257 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:43.478267 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.479791 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.479800 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:43.481745 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:43.481766 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.482893 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.482902 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:43.483154 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:43.484013 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:43.484042 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:43.487372 (+ 3330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:43.492600 (+ 5228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:43.492628 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:43.492638 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:43.493321 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:43.493851 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:43.493857 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:43.493861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:43.494053 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:43.494809 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:43.494819 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:43.495051 (+ 232us) client_negotiation.cc:769] Sending connection context
0811 00:26:43.495279 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 00:26:43.495676 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9578,"thread_start_us":10297,"threads_started":1}
I20260811 00:26:43.498651 9554 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.120.61:46301: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba"
member_type: NON_VOTER
last_known_addr {
host: "127.2.120.61"
port: 46301
}
attrs {
promote: true
}
}
dest_uuid: "97e4014ce0a74657ab9899f0ba4c2eee"
cas_config_opid_index: -1
I20260811 00:26:43.500110 9554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: NON_VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.120.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.60.keytab.
Entry for principal oryx/127.2.120.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.60.keytab.
I20260811 00:26:43.504537 10174 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5f58553db3fb46c49fa58a3ea3838cba (127.2.120.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: NON_VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: true } } }
I20260811 00:26:43.505937 10114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:43.506070 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.60:32855
--webserver_interface=127.2.120.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.60.keytab
--principal=oryx/127.2.120.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:45261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:33143,127.2.120.61:46301,127.2.120.60:32855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:26:43.508816 10174 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.120.61:46301
I20260811 00:26:43.510794 10174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: NON_VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: true } } }
I20260811 00:26:43.510931 10114 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 1 FOLLOWER]: Refusing update from remote peer 97e4014ce0a74657ab9899f0ba4c2eee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 00:26:43.511041 10174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:43.511045 10069 master.cc:537] Master@127.2.120.61:46301 shutting down...
I20260811 00:26:43.519762 10174 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: NON_VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 00:26:43.523172 9539 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260811 00:26:43.527343 10069 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:43.527966 10069 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:43.528287 10069 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: stopping tablet replica
I20260811 00:26:43.535864 10069 master.cc:552] Master@127.2.120.61:46301 shutdown complete.
W20260811 00:26:43.535871 9539 consensus_peers.cc:487] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee -> Peer 5f58553db3fb46c49fa58a3ea3838cba (127.2.120.61:46301): Couldn't send request to peer 5f58553db3fb46c49fa58a3ea3838cba. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260811 00:26:43.536226 10069 master_runner.cc:305] Clearing existing system tablet
I20260811 00:26:43.536849 10069 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:43.537048 10069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:43.540732 10069 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 00:26:43.542858 10181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:43.543309 10069 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:43.543848 10069 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 00:26:43.544027 10069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/wal
uuid: "5f58553db3fb46c49fa58a3ea3838cba"
format_stamp: "Formatted at 2026-08-11 00:26:43 on dist-test-slave-fd7l"
server_key: "27e367f878d6d12073e46b3e89bd05ee"
server_key_iv: "22c5d64ae0832a72f40973080d85e3ae"
server_key_version: "encryptionkey@0"
I20260811 00:26:43.549856 10069 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:43.558139 10069 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 00:26:43.558555 10069 log.cc:1196] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 00:26:43.559168 10069 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: Deleting consensus metadata
I20260811 00:26:43.559535 10069 master_runner.cc:315] Copying system tablet from 127.2.120.62:33143
I20260811 00:26:43.562661 10069 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: tablet copy: Beginning tablet copy session from remote peer at address 127.2.120.62:33143
I20260811 00:26:43.580574 10189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:43.563934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:43738)
0811 00:26:43.564625 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:43.564656 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:43.564910 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:43.565570 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:43.565579 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:43.566170 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:43.566466 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:43.566485 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.568041 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.568051 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:43.569513 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:43.569529 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.570551 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.570557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:43.570747 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:43.571570 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:43.571594 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:43.572748 (+ 1154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:43.577012 (+ 4264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:43.577018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:43.577022 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:43.577667 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:43.578343 (+ 676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:43.578349 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:43.578352 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:43.578569 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:43.579311 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:43.579316 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:43.579509 (+ 193us) client_negotiation.cc:769] Sending connection context
0811 00:26:43.579678 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 00:26:43.580088 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":620,"threads_started":1}
I20260811 00:26:43.580476 10146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:43.564293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43738 (local address 127.2.120.62:33143)
0811 00:26:43.564548 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:43.564555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:43.564846 (+ 291us) server_negotiation.cc:407] Connection header received
0811 00:26:43.565137 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:43.565145 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:43.565231 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:43.565309 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:43.566676 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.567837 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.569684 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.570359 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.570496 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:43.572995 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:43.573002 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:43.573008 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:43.573053 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:43.576811 (+ 3758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:43.577866 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:43.577875 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:43.577879 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:43.578172 (+ 293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:43.578723 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:43.578729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:43.578733 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:43.579154 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:43.579330 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:43.579823 (+ 493us) server_negotiation.cc:299] Negotiation successful
0811 00:26:43.580091 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 00:26:43.582257 9576 tablet_copy_service.cc:140] P 97e4014ce0a74657ab9899f0ba4c2eee: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5f58553db3fb46c49fa58a3ea3838cba ({username='oryx', principal='oryx/127.2.120.61@KRBTEST.COM'} at 127.0.0.1:43738)
I20260811 00:26:43.582752 9576 tablet_copy_service.cc:161] P 97e4014ce0a74657ab9899f0ba4c2eee: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5f58553db3fb46c49fa58a3ea3838cba at {username='oryx', principal='oryx/127.2.120.61@KRBTEST.COM'} at 127.0.0.1:43738: session id = 5f58553db3fb46c49fa58a3ea3838cba-00000000000000000000000000000000
I20260811 00:26:43.588757 9576 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 00:26:43.592165 10069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 00:26:43.599651 10069 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: tablet copy: Starting download of 0 data blocks...
I20260811 00:26:43.600116 10069 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: tablet copy: Starting download of 1 WAL segments...
I20260811 00:26:43.604476 10069 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 00:26:43.609483 9576 tablet_copy_service.cc:342] P 97e4014ce0a74657ab9899f0ba4c2eee: Request end of tablet copy session 5f58553db3fb46c49fa58a3ea3838cba-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.120.61@KRBTEST.COM'} at 127.0.0.1:43738
I20260811 00:26:43.609746 9576 tablet_copy_service.cc:433] P 97e4014ce0a74657ab9899f0ba4c2eee: ending tablet copy session 5f58553db3fb46c49fa58a3ea3838cba-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5f58553db3fb46c49fa58a3ea3838cba
I20260811 00:26:43.624616 10069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:43.628131 10191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:43.628885 10192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:43.637198 10196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:43.642385 10069 server_base.cc:1034] running on GCE node
I20260811 00:26:43.642838 10069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:43.643597 10069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:43.644862 10069 hybrid_clock.cc:648] HybridClock initialized: now 1786408003644795 us; error 66 us; skew 500 ppm
I20260811 00:26:43.645116 10069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:43 dist-test-slave-fd7l krb5kdc[9503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408003, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:43.647555 10069 init.cc:377] Logged in from keytab as oryx/127.2.120.61@KRBTEST.COM (short username oryx)
I20260811 00:26:43.651957 10069 webserver.cc:460] Webserver started at http://127.2.120.61:33245/ using document root <none> and password file <none>
I20260811 00:26:43.652377 10069 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:43.652470 10069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:43.656275 10069 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:26:43.658488 10203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:43.659020 10069 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 00:26:43.659159 10069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/wal
uuid: "5f58553db3fb46c49fa58a3ea3838cba"
format_stamp: "Formatted at 2026-08-11 00:26:43 on dist-test-slave-fd7l"
server_key: "27e367f878d6d12073e46b3e89bd05ee"
server_key_iv: "22c5d64ae0832a72f40973080d85e3ae"
server_key_version: "encryptionkey@0"
I20260811 00:26:43.659339 10069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:43.678464 10069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:43.683319 10069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:43.708158 10069 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.61:46301
I20260811 00:26:43.708184 10258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.61:46301 every 8 connection(s)
I20260811 00:26:43.709309 10069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-1/data/info.pb
I20260811 00:26:43.714963 10260 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:26:43.718897 10260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: Bootstrap starting.
I20260811 00:26:43.737296 10260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:26:43.737900 10260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: Bootstrap complete.
I20260811 00:26:43.739092 10260 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: NON_VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: true } }
I20260811 00:26:43.739380 10260 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f58553db3fb46c49fa58a3ea3838cba, State: Initialized, Role: LEARNER
I20260811 00:26:43.739650 10260 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: NON_VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: true } }
I20260811 00:26:43.740621 10263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: NON_VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: true } } }
I20260811 00:26:43.740968 10260 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:26:43.741098 10263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [sys.catalog]: This master's current role is: LEARNER
I20260811 00:26:43.751766 10274 catalog_manager.cc:1259] Loaded cluster ID: be18ea85b5ef4dc29a31e146ebce0964
I20260811 00:26:43.752041 10274 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: loading cluster ID for follower catalog manager: success
I20260811 00:26:43.756753 10274 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: acquiring CA information for follower catalog manager: success
I20260811 00:26:43.759709 10274 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:26:43.784910 10063 heartbeater.cc:503] Master 127.2.120.62:33143 was elected leader, sending a full tablet report...
W20260811 00:26:43.967541 10177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:43.967868 10177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:43.968031 10177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:43.976050 10177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:26:43.976219 10177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:43.976327 10177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:43.976438 10177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:26:43.976559 10177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:26:43.983641 10279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:43.975338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49006 (local address 127.2.120.61:46301)
0811 00:26:43.976115 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:43.976124 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:43.976144 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:26:43.976235 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:43.976243 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:43.976343 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:43.976508 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:43.977163 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.978550 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.981361 (+ 2811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.982507 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.982638 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:43.982904 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:43.983201 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 00:26:43.983316 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":481,"threads_started":1}
I20260811 00:26:43.983657 10171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:43.975101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:46301 (local address 127.0.0.1:49006)
0811 00:26:43.975315 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:43.975338 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:43.975519 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:43.976621 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:43.976626 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:43.976661 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:43.976960 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:43.976974 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.978740 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.978747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:43.981022 (+ 2275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:43.981044 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:43.982719 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:43.982726 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:43.982978 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:43.983048 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 00:26:43.983164 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 00:26:43.983351 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 00:26:43.985047 10233 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260811 00:26:43.985710 10263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97e4014ce0a74657ab9899f0ba4c2eee. Latest consensus state: current_term: 1 leader_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: NON_VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: true } } }
I20260811 00:26:43.986011 10263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [sys.catalog]: This master's current role is: LEARNER
I20260811 00:26:43.990075 10177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:45261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:33143,127.2.120.61:46301,127.2.120.60:32855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.60.keytab
--principal=oryx/127.2.120.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.60:32855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.120.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:43.991499 10177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:43.992949 10177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:44.016546 10282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:44.016582 10284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:44.016547 10281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:44.018337 10177 server_base.cc:1034] running on GCE node
I20260811 00:26:44.020817 10177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:44.025272 10177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:44.029138 10177 hybrid_clock.cc:648] HybridClock initialized: now 1786408004028877 us; error 170 us; skew 500 ppm
I20260811 00:26:44.030442 10177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:44 dist-test-slave-fd7l krb5kdc[9503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408004, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:44.038187 10177 init.cc:377] Logged in from keytab as oryx/127.2.120.60@KRBTEST.COM (short username oryx)
I20260811 00:26:44.042091 10177 webserver.cc:460] Webserver started at http://127.2.120.60:33657/ using document root <none> and password file <none>
I20260811 00:26:44.043898 10177 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:44.044234 10177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:44.044992 10177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:44.048640 10177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/data/instance:
uuid: "e60785b019bf4273960830aeeb93fd30"
format_stamp: "Formatted at 2026-08-11 00:26:44 on dist-test-slave-fd7l"
server_key: "b3f33537810e00654ef06352cac632db"
server_key_iv: "03cf0c213fdf5ca0bbae5956bc8a843a"
server_key_version: "encryptionkey@0"
I20260811 00:26:44.049782 10177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/wal/instance:
uuid: "e60785b019bf4273960830aeeb93fd30"
format_stamp: "Formatted at 2026-08-11 00:26:44 on dist-test-slave-fd7l"
server_key: "b3f33537810e00654ef06352cac632db"
server_key_iv: "03cf0c213fdf5ca0bbae5956bc8a843a"
server_key_version: "encryptionkey@0"
I20260811 00:26:44.057397 10177 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260811 00:26:44.062675 10291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:44.064970 10177 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:26:44.065213 10177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/wal
uuid: "e60785b019bf4273960830aeeb93fd30"
format_stamp: "Formatted at 2026-08-11 00:26:44 on dist-test-slave-fd7l"
server_key: "b3f33537810e00654ef06352cac632db"
server_key_iv: "03cf0c213fdf5ca0bbae5956bc8a843a"
server_key_version: "encryptionkey@0"
I20260811 00:26:44.065452 10177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:44.101362 10177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:44.106353 10177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:44.106698 10177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:44.128767 10177 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.60:32855
I20260811 00:26:44.128803 10352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.60:32855 every 8 connection(s)
I20260811 00:26:44.130265 10177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/data/info.pb
I20260811 00:26:44.132014 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 10177
I20260811 00:26:44.134918 10353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:44.142680 10353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:26:44 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.120.60@KRBTEST.COM
Aug 11 00:26:44 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408004, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.60@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM
I20260811 00:26:44.159340 10356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.134327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36616 (local address 127.2.120.60:32855)
0811 00:26:44.135279 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:44.135292 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:44.135334 (+ 42us) server_negotiation.cc:407] Connection header received
0811 00:26:44.137309 (+ 1975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:44.137334 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:44.137742 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:44.138178 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:44.139741 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.141265 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.142718 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.143445 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.143654 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.149435 (+ 5781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:44.149445 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:44.149456 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:44.149503 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:44.153634 (+ 4131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.154662 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.154676 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.154683 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.154878 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.155309 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.155314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.155318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.155991 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:44.156323 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:44.157077 (+ 754us) server_negotiation.cc:299] Negotiation successful
0811 00:26:44.157591 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":748,"thread_start_us":459,"threads_started":1}
I20260811 00:26:44.165374 10357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.145826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43742 (local address 127.2.120.62:33143)
0811 00:26:44.147338 (+ 1512us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:44.147343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:44.147357 (+ 14us) server_negotiation.cc:407] Connection header received
0811 00:26:44.147408 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:44.147412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:44.147470 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:44.147599 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:44.149059 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.150168 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.152647 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.153117 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.153237 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.157545 (+ 4308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:44.157552 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:44.157566 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:44.157613 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:44.161475 (+ 3862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.162642 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.162647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.162650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.162868 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.163688 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.163692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.163695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.164000 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:44.164145 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:44.164795 (+ 650us) server_negotiation.cc:299] Negotiation successful
0811 00:26:44.164996 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1412,"thread_start_us":1321,"threads_started":1}
I20260811 00:26:44.165390 10358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.145797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:43742)
0811 00:26:44.146732 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:44.146787 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:44.146951 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:44.147683 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:44.147691 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:44.148598 (+ 907us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:44.148869 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:44.148879 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.150355 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.150363 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:44.152356 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:44.152386 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.153347 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.153356 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:44.153577 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.154460 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:44.154487 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:44.157311 (+ 2824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:44.161645 (+ 4334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:44.161656 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:44.161663 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:44.162464 (+ 801us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:44.163158 (+ 694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:44.163166 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:44.163171 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:44.163499 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:44.164190 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:44.164199 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:44.164401 (+ 202us) client_negotiation.cc:769] Sending connection context
0811 00:26:44.164680 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 00:26:44.164986 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":436,"threads_started":1}
I20260811 00:26:44.167055 10353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:26:44 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408004, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.60@KRBTEST.COM for oryx/127.2.120.61@KRBTEST.COM
I20260811 00:26:44.182312 10358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.167871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:46301 (local address 127.0.0.1:49020)
0811 00:26:44.168134 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:44.168157 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:44.168316 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:44.168713 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:44.168717 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:44.169128 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:44.169278 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:44.169287 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.170322 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.170328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:44.171342 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:44.171354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.172137 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.172143 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:44.172349 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.173090 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:44.173108 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:44.175420 (+ 2312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:44.179540 (+ 4120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:44.179547 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:44.179551 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:44.180016 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:44.180450 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:44.180458 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:44.180462 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:44.180690 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:44.181251 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:44.181256 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:44.181417 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 00:26:44.181574 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 00:26:44.181919 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 00:26:44.182323 10279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.167913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49020 (local address 127.2.120.61:46301)
0811 00:26:44.168129 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:44.168136 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:44.168242 (+ 106us) server_negotiation.cc:407] Connection header received
0811 00:26:44.168443 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:44.168456 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:44.168571 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:44.168686 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:44.169436 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.170143 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.171508 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.171955 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.172109 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.175696 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:44.175701 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:44.175704 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:44.175741 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:44.179304 (+ 3563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.180159 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.180166 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.180169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.180320 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.180809 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.180815 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.180818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.181124 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:44.181294 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:44.181682 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 00:26:44.181924 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 00:26:44.183787 10353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:26:44 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408004, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.60@KRBTEST.COM for oryx/127.2.120.60@KRBTEST.COM
I20260811 00:26:44.198319 10358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.184660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:32855 (local address 127.0.0.1:36618)
0811 00:26:44.184850 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:44.184892 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:44.185034 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:44.185399 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:44.185403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:44.185854 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:44.186017 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:44.186027 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.187136 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.187144 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:44.188162 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:44.188176 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.189029 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.189035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:44.189263 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.189908 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:44.189927 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:44.192331 (+ 2404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:44.195533 (+ 3202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:44.195540 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:44.195543 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:44.195916 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:44.196399 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:44.196405 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:44.196409 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:44.196645 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:44.197182 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:44.197186 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:44.197401 (+ 215us) client_negotiation.cc:769] Sending connection context
0811 00:26:44.197556 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 00:26:44.197863 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 00:26:44.198352 10356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.184839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36618 (local address 127.2.120.60:32855)
0811 00:26:44.184997 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:44.185003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:44.185022 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:26:44.185171 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:44.185177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:44.185268 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:44.185381 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:44.186171 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.186994 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.188314 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.188838 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.189000 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.192623 (+ 3623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:44.192630 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:44.192641 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:44.192672 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:44.195374 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.196052 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.196057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.196060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.196265 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.196752 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.196759 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.196762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.197050 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:44.197212 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:44.197643 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 00:26:44.198026 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":149,"server-negotiator.queue_time_us":56}
I20260811 00:26:44.203989 10353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: Bootstrap starting.
I20260811 00:26:44.207752 10353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:44.209113 10353 log.cc:824] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:44.212775 10353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: No bootstrap required, opened a new log
I20260811 00:26:44.222227 10353 raft_consensus.cc:348] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } }
I20260811 00:26:44.222905 10353 raft_consensus.cc:374] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:44.223075 10353 raft_consensus.cc:729] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e60785b019bf4273960830aeeb93fd30, State: Initialized, Role: FOLLOWER
I20260811 00:26:44.223803 10353 consensus_queue.cc:260] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } }
I20260811 00:26:44.224895 10360 sys_catalog.cc:455] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } } }
I20260811 00:26:44.225318 10360 sys_catalog.cc:458] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:26:44.226315 10353 sys_catalog.cc:564] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 00:26:44.238530 10373 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:26:44.238773 10373 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:26:44.241973 10177 master_runner.cc:428] Detected that this master e60785b019bf4273960830aeeb93fd30 is joining an existing cluster
I20260811 00:26:44.242121 10177 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.120.60:32855
I20260811 00:26:44.267808 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 9622
I20260811 00:26:44.269452 10357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.247390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43752 (local address 127.2.120.62:33143)
0811 00:26:44.247631 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:44.247639 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:44.249245 (+ 1606us) server_negotiation.cc:407] Connection header received
0811 00:26:44.249626 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:44.249633 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:44.249756 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:44.249847 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:44.252241 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.253761 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.256170 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.256838 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.257003 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.260426 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:44.260434 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:44.260441 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:44.260488 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:44.265079 (+ 4591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.266032 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.266038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.266042 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.266230 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.266838 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.266846 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.266851 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.267268 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:44.267405 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:44.268229 (+ 824us) server_negotiation.cc:299] Negotiation successful
0811 00:26:44.268536 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 00:26:44.271358 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.246829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:43752)
0811 00:26:44.249027 (+ 2198us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:44.249049 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:44.249369 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:44.250172 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:44.250180 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:44.251299 (+ 1119us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:44.251826 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:44.251844 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.254048 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.254053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:44.255983 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:44.255999 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.257285 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.257294 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:44.257520 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.258387 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:44.258418 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:44.260130 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:44.265276 (+ 5146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:44.265281 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:44.265284 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:44.265868 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:44.266394 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:44.266401 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:44.266407 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:44.266682 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:44.267508 (+ 826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:44.267516 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:44.267753 (+ 237us) client_negotiation.cc:769] Sending connection context
0811 00:26:44.270025 (+ 2272us) client_negotiation.cc:240] Negotiation successful
0811 00:26:44.270824 (+ 799us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2050,"thread_start_us":1881,"threads_started":1}
I20260811 00:26:44.279384 10279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.246955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49024 (local address 127.2.120.61:46301)
0811 00:26:44.247165 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:44.247173 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:44.253909 (+ 6736us) server_negotiation.cc:407] Connection header received
0811 00:26:44.254916 (+ 1007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:44.254923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:44.255054 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:44.255176 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:44.257102 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.258240 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.260972 (+ 2732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.261635 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.261753 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.265738 (+ 3985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:44.265745 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:44.265749 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:44.265797 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:44.269690 (+ 3893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.275648 (+ 5958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.275660 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.275666 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.275920 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.276799 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.276808 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.276814 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.277333 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:44.277524 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:44.278420 (+ 896us) server_negotiation.cc:299] Negotiation successful
0811 00:26:44.278816 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:26:44.280941 10356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.249782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36634 (local address 127.2.120.60:32855)
0811 00:26:44.249957 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:44.249964 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:44.257269 (+ 7305us) server_negotiation.cc:407] Connection header received
0811 00:26:44.257520 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:44.257526 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:44.257596 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:44.257672 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:44.259643 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.261081 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.263800 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.264683 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.264806 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.268142 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:44.268149 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:44.268156 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:44.268191 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:44.272302 (+ 4111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.273242 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.273248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.273252 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.273427 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.273879 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.273885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.273888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.274353 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:44.274513 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:44.275136 (+ 623us) server_negotiation.cc:299] Negotiation successful
0811 00:26:44.280434 (+ 5298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":41}
I20260811 00:26:44.281883 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.246960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:46301 (local address 127.0.0.1:49024)
0811 00:26:44.253771 (+ 6811us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:44.253794 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:44.254732 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:44.255191 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:44.255195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:44.256137 (+ 942us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:44.256496 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:44.256535 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.258453 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.258458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:44.260703 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:44.260723 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.262118 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.262125 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:44.262513 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.263780 (+ 1267us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:44.263820 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:44.265428 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:44.269950 (+ 4522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:44.269960 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:44.269967 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:44.272232 (+ 2265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:44.276247 (+ 4015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:44.276261 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:44.276268 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:44.276594 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:44.277640 (+ 1046us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:44.277648 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:44.278010 (+ 362us) client_negotiation.cc:769] Sending connection context
0811 00:26:44.280073 (+ 2063us) client_negotiation.cc:240] Negotiation successful
0811 00:26:44.280750 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6594,"mutex_wait_us":251,"thread_start_us":9773,"threads_started":1}
I20260811 00:26:44.282303 10381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.249306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:32855 (local address 127.0.0.1:36634)
0811 00:26:44.257167 (+ 7861us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:44.257190 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:44.257365 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:44.257901 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:44.257907 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:44.258981 (+ 1074us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:44.259416 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:44.259429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.261345 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.261352 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:44.263596 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:44.263616 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.265184 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.265197 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:44.265389 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.266059 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:44.266117 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:44.267958 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:44.272442 (+ 4484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:44.272447 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:44.272452 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:44.273093 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:44.273581 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:44.273586 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:44.273590 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:44.273745 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:44.274609 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:44.274614 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:44.274788 (+ 174us) client_negotiation.cc:769] Sending connection context
0811 00:26:44.281245 (+ 6457us) client_negotiation.cc:240] Negotiation successful
0811 00:26:44.281615 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7618,"thread_start_us":4035,"threads_started":1}
Aug 11 00:26:44 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408000, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.62@KRBTEST.COM for oryx/127.2.120.60@KRBTEST.COM
Aug 11 00:26:44 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.120.1@KRBTEST.COM
I20260811 00:26:44.301182 10171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.278834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:32855 (local address 127.0.0.1:36650)
0811 00:26:44.279274 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:44.279316 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:44.279451 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:44.282385 (+ 2934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:44.282390 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:44.282932 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:44.283191 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:44.283206 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.285238 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.285245 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:44.286586 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:44.286607 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.287859 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.287869 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:44.288119 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.289065 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:44.289091 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:44.292126 (+ 3035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:44.297560 (+ 5434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:44.297567 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:44.297573 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:44.298377 (+ 804us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:44.299007 (+ 630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:44.299013 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:44.299017 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:44.299221 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:44.299975 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:44.299980 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:44.300175 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 00:26:44.300362 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 00:26:44.300743 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299}
I20260811 00:26:44.301240 10385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.279681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36650 (local address 127.2.120.60:32855)
0811 00:26:44.281023 (+ 1342us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:44.281029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:44.281054 (+ 25us) server_negotiation.cc:407] Connection header received
0811 00:26:44.281164 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:44.281169 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:44.281271 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:44.283405 (+ 2134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:44.283861 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.285088 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.286789 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.287607 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.288406 (+ 799us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.292398 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:44.292404 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:44.292407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:44.292456 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:44.297272 (+ 4816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.298552 (+ 1280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.298563 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.298567 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.298811 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.299377 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.299383 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.299386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.299802 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:44.299998 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:44.300489 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 00:26:44.300774 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1177,"thread_start_us":1438,"threads_started":1}
I20260811 00:26:44.302824 9554 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.120.60:32855: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e60785b019bf4273960830aeeb93fd30"
member_type: NON_VOTER
last_known_addr {
host: "127.2.120.60"
port: 32855
}
attrs {
promote: true
}
}
dest_uuid: "97e4014ce0a74657ab9899f0ba4c2eee"
cas_config_opid_index: 5
I20260811 00:26:44.304009 9554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: NON_VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: true } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: NON_VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: true } }
I20260811 00:26:44.306362 10174 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e60785b019bf4273960830aeeb93fd30 (127.2.120.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: NON_VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: true } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: NON_VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: true } } }
I20260811 00:26:44.310482 10384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.269313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41824 (local address 127.2.120.1:38089)
0811 00:26:44.285124 (+ 15811us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:44.285133 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:44.285194 (+ 61us) server_negotiation.cc:407] Connection header received
0811 00:26:44.285271 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:44.285281 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:44.285607 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:44.285819 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:44.287869 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.289635 (+ 1766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.291971 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.293039 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.293206 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.299740 (+ 6534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:44.299765 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:44.299792 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:44.299843 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:44.305148 (+ 5305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.306450 (+ 1302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.306462 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.306466 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.306731 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.307280 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.307286 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.307289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.307839 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:44.308170 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:44.309691 (+ 1521us) server_negotiation.cc:299] Negotiation successful
0811 00:26:44.310021 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15613,"thread_start_us":15839,"threads_started":1}
I20260811 00:26:44.310789 10233 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 1 LEARNER]: Refusing update from remote peer 97e4014ce0a74657ab9899f0ba4c2eee: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 00:26:44.311646 10386 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: NON_VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 00:26:44.312194 9763 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:44.312570 10322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:44.322611 10174 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.120.60:32855
I20260811 00:26:44.326398 10386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: NON_VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: true } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: NON_VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: true } } }
I20260811 00:26:44.327154 10386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:44.335011 10233 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e60785b019bf4273960830aeeb93fd30 (127.2.120.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: NON_VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: true } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: NON_VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: true } } }
I20260811 00:26:44.343678 10263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: NON_VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: true } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: NON_VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: true } } }
I20260811 00:26:44.344496 10263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [sys.catalog]: This master's current role is: LEARNER
I20260811 00:26:44.346392 10174 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee: attempting to promote NON_VOTER 5f58553db3fb46c49fa58a3ea3838cba to VOTER
I20260811 00:26:44.347507 10174 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: NON_VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: true } }
I20260811 00:26:44.350962 10232 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 1 LEARNER]: Refusing update from remote peer 97e4014ce0a74657ab9899f0ba4c2eee: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 00:26:44.351811 10174 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 00:26:44.357530 10387 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5f58553db3fb46c49fa58a3ea3838cba (127.2.120.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: NON_VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: true } } }
I20260811 00:26:44.360999 10177 master.cc:537] Master@127.2.120.60:32855 shutting down...
I20260811 00:26:44.361244 10322 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 1 FOLLOWER]: Refusing update from remote peer 97e4014ce0a74657ab9899f0ba4c2eee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20260811 00:26:44.361296 10298 tablet.cc:2369] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:26:44.361793 10321 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 1 FOLLOWER]: Refusing update from remote peer 97e4014ce0a74657ab9899f0ba4c2eee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 00:26:44.365051 10387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: NON_VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: true } } }
I20260811 00:26:44.365885 10387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:44.366294 10387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: NON_VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: true } } }
I20260811 00:26:44.366674 10387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:44.368969 10233 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5f58553db3fb46c49fa58a3ea3838cba (127.2.120.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: NON_VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: true } } }
I20260811 00:26:44.375008 10263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: NON_VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: true } } }
I20260811 00:26:44.375339 10263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:26:44.369138 10387 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: NON_VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.007s
I20260811 00:26:44.385421 10177 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:44.386029 10177 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:44.386297 10177 tablet_replica.cc:330] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: stopping tablet replica
W20260811 00:26:44.388680 9537 consensus_peers.cc:487] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee -> Peer e60785b019bf4273960830aeeb93fd30 (127.2.120.60:32855): Couldn't send request to peer e60785b019bf4273960830aeeb93fd30. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 00:26:44.404896 10177 master.cc:552] Master@127.2.120.60:32855 shutdown complete.
I20260811 00:26:44.405042 10177 master_runner.cc:305] Clearing existing system tablet
I20260811 00:26:44.405507 10177 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:44.405625 10177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:44.409294 10177 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 00:26:44.412981 10393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:44.413483 10177 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:44.413774 10177 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 00:26:44.413893 10177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/wal
uuid: "e60785b019bf4273960830aeeb93fd30"
format_stamp: "Formatted at 2026-08-11 00:26:44 on dist-test-slave-fd7l"
server_key: "b3f33537810e00654ef06352cac632db"
server_key_iv: "03cf0c213fdf5ca0bbae5956bc8a843a"
server_key_version: "encryptionkey@0"
I20260811 00:26:44.419585 10177 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:44.427888 10177 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 00:26:44.428126 10177 log.cc:1196] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 00:26:44.428730 10177 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: Deleting consensus metadata
I20260811 00:26:44.429067 10177 master_runner.cc:315] Copying system tablet from 127.2.120.62:33143
I20260811 00:26:44.438351 10177 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: tablet copy: Beginning tablet copy session from remote peer at address 127.2.120.62:33143
I20260811 00:26:44.457897 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 9622
I20260811 00:26:44.459544 10401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.439321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:43760)
0811 00:26:44.439930 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:44.439955 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:44.440141 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:44.440625 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:44.440632 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:44.441295 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:44.441607 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:44.441622 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.443650 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.443657 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:44.445320 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:44.445343 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.446498 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.446504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:44.446716 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.447579 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:44.447606 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:44.448913 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:44.453954 (+ 5041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:44.453964 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:44.453971 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:44.454687 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:44.455824 (+ 1137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:44.455829 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:44.455859 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:44.456341 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:44.457187 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:44.457195 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:44.457467 (+ 272us) client_negotiation.cc:769] Sending connection context
0811 00:26:44.457700 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 00:26:44.458974 (+ 1274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":316,"threads_started":1}
I20260811 00:26:44.460196 10357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.439434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43760 (local address 127.2.120.62:33143)
0811 00:26:44.439687 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:44.439696 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:44.440086 (+ 390us) server_negotiation.cc:407] Connection header received
0811 00:26:44.440307 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:44.440313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:44.440402 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:44.440537 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:44.441830 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.443443 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.445529 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.446264 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.446421 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.449229 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:44.449237 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:44.449242 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:44.449289 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:44.453692 (+ 4403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.455381 (+ 1689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.455390 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.455397 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.455618 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:44.456547 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:44.456553 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:44.456556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:44.457021 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:44.457211 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:44.457992 (+ 781us) server_negotiation.cc:299] Negotiation successful
0811 00:26:44.458302 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 00:26:44.460819 9576 tablet_copy_service.cc:140] P 97e4014ce0a74657ab9899f0ba4c2eee: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e60785b019bf4273960830aeeb93fd30 ({username='oryx', principal='oryx/127.2.120.60@KRBTEST.COM'} at 127.0.0.1:43760)
I20260811 00:26:44.461040 9576 tablet_copy_service.cc:161] P 97e4014ce0a74657ab9899f0ba4c2eee: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e60785b019bf4273960830aeeb93fd30 at {username='oryx', principal='oryx/127.2.120.60@KRBTEST.COM'} at 127.0.0.1:43760: session id = e60785b019bf4273960830aeeb93fd30-00000000000000000000000000000000
I20260811 00:26:44.467000 9576 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 00:26:44.469748 10177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 00:26:44.477316 10177 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: tablet copy: Starting download of 0 data blocks...
I20260811 00:26:44.477672 10177 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: tablet copy: Starting download of 1 WAL segments...
I20260811 00:26:44.480881 10177 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 00:26:44.486006 9576 tablet_copy_service.cc:342] P 97e4014ce0a74657ab9899f0ba4c2eee: Request end of tablet copy session e60785b019bf4273960830aeeb93fd30-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.120.60@KRBTEST.COM'} at 127.0.0.1:43760
I20260811 00:26:44.486202 9576 tablet_copy_service.cc:433] P 97e4014ce0a74657ab9899f0ba4c2eee: ending tablet copy session e60785b019bf4273960830aeeb93fd30-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e60785b019bf4273960830aeeb93fd30
I20260811 00:26:44.487911 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:38089
--local_ip_for_outbound_sockets=127.2.120.1
--tserver_master_addrs=127.2.120.62:33143
--webserver_port=36037
--webserver_interface=127.2.120.1
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.1.keytab
--principal=oryx/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:45261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.120.62:33143,127.2.120.61:46301
--tserver_master_addrs=127.2.120.62:33143,127.2.120.61:46301,127.2.120.60:32855 with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:26:44.500981 10177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:44.504319 10403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:44.504829 10404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:44.512070 10406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:44.514021 10177 server_base.cc:1034] running on GCE node
I20260811 00:26:44.514479 10177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:44.515224 10177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:44.516623 10177 hybrid_clock.cc:648] HybridClock initialized: now 1786408004516460 us; error 52 us; skew 500 ppm
I20260811 00:26:44.517086 10177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:44 dist-test-slave-fd7l krb5kdc[9503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408004, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:44.519629 10177 init.cc:377] Logged in from keytab as oryx/127.2.120.60@KRBTEST.COM (short username oryx)
I20260811 00:26:44.529814 10177 webserver.cc:460] Webserver started at http://127.2.120.60:35491/ using document root <none> and password file <none>
I20260811 00:26:44.530207 10177 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:44.530323 10177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:44.533941 10177 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260811 00:26:44.536294 10413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:44.536924 10177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 00:26:44.537071 10177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/wal
uuid: "e60785b019bf4273960830aeeb93fd30"
format_stamp: "Formatted at 2026-08-11 00:26:44 on dist-test-slave-fd7l"
server_key: "b3f33537810e00654ef06352cac632db"
server_key_iv: "03cf0c213fdf5ca0bbae5956bc8a843a"
server_key_version: "encryptionkey@0"
I20260811 00:26:44.537281 10177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:44.557686 10177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:44.562745 10177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:44.601595 10177 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.60:32855
I20260811 00:26:44.601643 10467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.60:32855 every 8 connection(s)
I20260811 00:26:44.602741 10177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/master-2/data/info.pb
I20260811 00:26:44.610678 10468 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:26:44.613569 10468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: Bootstrap starting.
I20260811 00:26:44.631381 10468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:26:44.632007 10468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: Bootstrap complete.
I20260811 00:26:44.633144 10468 raft_consensus.cc:348] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: NON_VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: true } }
I20260811 00:26:44.633486 10468 raft_consensus.cc:729] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e60785b019bf4273960830aeeb93fd30, State: Initialized, Role: LEARNER
I20260811 00:26:44.633800 10468 consensus_queue.cc:260] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: NON_VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: true } }
I20260811 00:26:44.635140 10468 sys_catalog.cc:564] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:26:44.634729 10472 sys_catalog.cc:455] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: NON_VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: true } } }
I20260811 00:26:44.635483 10472 sys_catalog.cc:458] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [sys.catalog]: This master's current role is: LEARNER
I20260811 00:26:44.650063 10484 catalog_manager.cc:1259] Loaded cluster ID: be18ea85b5ef4dc29a31e146ebce0964
I20260811 00:26:44.650231 10484 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: loading cluster ID for follower catalog manager: success
I20260811 00:26:44.653565 10484 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: acquiring CA information for follower catalog manager: success
I20260811 00:26:44.656652 10484 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:26:44.803939 10171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.793965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:32855 (local address 127.0.0.1:36660)
0811 00:26:44.794342 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:44.794370 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:44.794538 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:44.795876 (+ 1338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:44.795882 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:44.795919 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:44.796139 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:44.796153 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.798395 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.798406 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:44.800821 (+ 2415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:44.800851 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.802861 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.802869 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:44.803235 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.803309 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 00:26:44.803445 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 00:26:44.803631 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201}
I20260811 00:26:44.804087 10485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:44.794251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36660 (local address 127.2.120.60:32855)
0811 00:26:44.795330 (+ 1079us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:44.795340 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:44.795372 (+ 32us) server_negotiation.cc:407] Connection header received
0811 00:26:44.795474 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:44.795483 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:44.795620 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:44.795810 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:44.796383 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.798172 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.801099 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:44.802567 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:44.802757 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:44.803041 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:44.803507 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 00:26:44.803673 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":873,"thread_start_us":607,"threads_started":1}
I20260811 00:26:44.805554 10442 raft_consensus.cc:1203] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260811 00:26:44.806510 10472 sys_catalog.cc:455] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97e4014ce0a74657ab9899f0ba4c2eee. Latest consensus state: current_term: 1 leader_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: NON_VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: true } } }
I20260811 00:26:44.806844 10472 sys_catalog.cc:458] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [sys.catalog]: This master's current role is: LEARNER
W20260811 00:26:44.945457 10402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:44.945845 10402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:44.946031 10402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:44.954250 10402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:44.954450 10402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:44.954666 10402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:26:44.968086 10402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:45261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.1.keytab
--principal=oryx/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:38089
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=36037
--tserver_master_addrs=127.2.120.62:33143,127.2.120.61:46301,127.2.120.60:32855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:44.969659 10402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:44.971094 10402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:44.986987 10491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:44.986965 10492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:44.987510 10494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:44.988240 10402 server_base.cc:1034] running on GCE node
I20260811 00:26:44.989070 10402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:44.990239 10402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:44.991499 10402 hybrid_clock.cc:648] HybridClock initialized: now 1786408004991444 us; error 37 us; skew 500 ppm
I20260811 00:26:44.991922 10402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:44 dist-test-slave-fd7l krb5kdc[9503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408004, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:44.995820 10402 init.cc:377] Logged in from keytab as oryx/127.2.120.1@KRBTEST.COM (short username oryx)
I20260811 00:26:44.998449 10402 webserver.cc:460] Webserver started at http://127.2.120.1:36037/ using document root <none> and password file <none>
I20260811 00:26:44.999397 10402 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:44.999627 10402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:45.006436 10402 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:26:45.009893 10503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:45.011627 10402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:26:45.011881 10402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "0dcaa36e9c34456da1814e7cc9e27ace"
format_stamp: "Formatted at 2026-08-11 00:26:41 on dist-test-slave-fd7l"
server_key: "a5b096ad16374b1ae13bd2104b39664b"
server_key_iv: "086c77158a1a6f590a028c2e3cfc30ac"
server_key_version: "encryptionkey@0"
I20260811 00:26:45.013186 10402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:45.031085 10402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:45.036944 10402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:45.037281 10402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:45.038616 10402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:45.040624 10402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:45.040733 10402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:45.040870 10402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:45.040951 10402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:45.083144 10402 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:38089
I20260811 00:26:45.083271 10626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:38089 every 8 connection(s)
I20260811 00:26:45.084826 10402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:26:45.085965 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 10402
I20260811 00:26:45.086397 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 9774
Aug 11 00:26:45 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408004, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.60@KRBTEST.COM
Aug 11 00:26:45 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408004, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM
I20260811 00:26:45.127017 10485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:45.093986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:52251 (local address 127.2.120.60:32855)
0811 00:26:45.094625 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:45.094635 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:45.099698 (+ 5063us) server_negotiation.cc:407] Connection header received
0811 00:26:45.104601 (+ 4903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:45.104610 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:45.104736 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:45.104825 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:45.107633 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.109138 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.111449 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.112145 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.112289 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:45.117258 (+ 4969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:45.117266 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:45.117270 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:45.117321 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:45.121219 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:45.122461 (+ 1242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:45.122467 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:45.122471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:45.122635 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:45.123611 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:45.123617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:45.123621 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:45.124047 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:45.124190 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:45.126299 (+ 2109us) server_negotiation.cc:299] Negotiation successful
0811 00:26:45.126577 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469}
Aug 11 00:26:45 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408004, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.61@KRBTEST.COM
I20260811 00:26:45.129573 10635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:45.093104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:32855 (local address 127.2.120.1:52251)
0811 00:26:45.099543 (+ 6439us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:45.099559 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:45.104022 (+ 4463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:45.105519 (+ 1497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:45.105530 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:45.106366 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:45.107249 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:45.107275 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.109442 (+ 2167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.109448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:45.111228 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:45.111247 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.112774 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.112784 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:45.113075 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:45.113863 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:45.113895 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:45.116825 (+ 2930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:45.121376 (+ 4551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:45.121386 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:45.121406 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:45.122284 (+ 878us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:45.122797 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:45.122804 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:45.122810 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:45.123440 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:45.124279 (+ 839us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:45.124288 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:45.125495 (+ 1207us) client_negotiation.cc:769] Sending connection context
0811 00:26:45.125914 (+ 419us) client_negotiation.cc:240] Negotiation successful
0811 00:26:45.127047 (+ 1133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6281,"thread_start_us":6039,"threads_started":1}
I20260811 00:26:45.131656 10627 heartbeater.cc:347] Connected to a master server at 127.2.120.60:32855
Aug 11 00:26:45 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.120.2@KRBTEST.COM
I20260811 00:26:45.132050 10627 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:45.133111 10627 heartbeater.cc:511] Master 127.2.120.60:32855 requested a full tablet report, sending...
I20260811 00:26:45.135962 10432 ts_manager.cc:194] Registered new tserver with Master: 0dcaa36e9c34456da1814e7cc9e27ace (127.2.120.1:38089)
I20260811 00:26:45.136713 10633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:45.088391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:39181 (local address 127.2.120.62:33143)
0811 00:26:45.109420 (+ 21029us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:45.109427 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:45.109448 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:26:45.109516 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:45.109521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:45.109644 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:45.109769 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:45.113533 (+ 3764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.117759 (+ 4226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.121289 (+ 3530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.121941 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.122084 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:45.126500 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:45.126509 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:45.126514 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:45.126568 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:45.130714 (+ 4146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:45.132653 (+ 1939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:45.132660 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:45.132664 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:45.132834 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:45.133663 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:45.133669 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:45.133673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:45.135338 (+ 1665us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:45.135521 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:45.135922 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 00:26:45.136266 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20869,"thread_start_us":21522,"threads_started":1}
I20260811 00:26:45.136772 10634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:45.089328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.1:39181)
0811 00:26:45.092063 (+ 2735us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:45.099536 (+ 7473us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:45.104022 (+ 4486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:45.111508 (+ 7486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:45.111514 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:45.112111 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:45.112715 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:45.112732 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.117907 (+ 5175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.117916 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:45.120423 (+ 2507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:45.120441 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.122244 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.122254 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:45.122473 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:45.123280 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:45.123303 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:45.126097 (+ 2794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:45.131386 (+ 5289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:45.131392 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:45.131397 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:45.131936 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:45.133355 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:45.133360 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:45.133364 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:45.133524 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:45.135476 (+ 1952us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:45.135482 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:45.135642 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 00:26:45.135787 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 00:26:45.136225 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2264,"thread_start_us":1975,"threads_started":1}
I20260811 00:26:45.138022 10629 heartbeater.cc:347] Connected to a master server at 127.2.120.62:33143
I20260811 00:26:45.138175 10629 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:45.138818 10629 heartbeater.cc:511] Master 127.2.120.62:33143 requested a full tablet report, sending...
I20260811 00:26:45.140326 9554 ts_manager.cc:194] Re-registered known tserver with Master: 0dcaa36e9c34456da1814e7cc9e27ace (127.2.120.1:38089)
I20260811 00:26:45.140756 10636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:45.093205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:46301 (local address 127.2.120.1:57705)
0811 00:26:45.117486 (+ 24281us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:45.117506 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:45.117649 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:45.118140 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:45.118145 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:45.119272 (+ 1127us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:45.119665 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:45.119680 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.122310 (+ 2630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.122318 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:45.124226 (+ 1908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:45.124241 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.125595 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.125608 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:45.125840 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:45.126503 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:45.126528 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:45.130185 (+ 3657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:45.137162 (+ 6977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:45.137168 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:45.137172 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:45.137786 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:45.138462 (+ 676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:45.138470 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:45.138474 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:45.138670 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:45.139486 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:45.139491 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:45.139695 (+ 204us) client_negotiation.cc:769] Sending connection context
0811 00:26:45.139862 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 00:26:45.140210 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24159,"thread_start_us":23962,"threads_started":1}
I20260811 00:26:45.140874 10638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:45.094229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:57705 (local address 127.2.120.61:46301)
0811 00:26:45.116917 (+ 22688us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:45.116926 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:45.117782 (+ 856us) server_negotiation.cc:407] Connection header received
0811 00:26:45.117854 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:45.117862 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:45.117971 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:45.118109 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:45.119884 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.121057 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.124400 (+ 3343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.125173 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.125309 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:45.130499 (+ 5190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:45.130508 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:45.130512 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:45.130570 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:45.137003 (+ 6433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:45.137974 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:45.137984 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:45.137987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:45.138304 (+ 317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:45.138790 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:45.138795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:45.138799 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:45.139336 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:45.139504 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:45.140045 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 00:26:45.140274 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22552,"thread_start_us":22584,"threads_started":1}
I20260811 00:26:45.141633 9554 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:39181
I20260811 00:26:45.142226 10630 heartbeater.cc:347] Connected to a master server at 127.2.120.61:46301
I20260811 00:26:45.142469 10630 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:45.143006 10632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:45.087941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48944 (local address 127.2.120.2:45043)
0811 00:26:45.115117 (+ 27176us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:45.115131 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:45.116097 (+ 966us) server_negotiation.cc:407] Connection header received
0811 00:26:45.116321 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:45.116333 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:45.116702 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:45.116880 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:45.120494 (+ 3614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.122152 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.124314 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.125375 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.125498 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:45.133059 (+ 7561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:45.133081 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:45.133103 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:45.133153 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:45.137796 (+ 4643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:45.139152 (+ 1356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:45.139164 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:45.139167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:45.139522 (+ 355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:45.140070 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:45.140076 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:45.140079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:45.140659 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:45.141196 (+ 537us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:45.142001 (+ 805us) server_negotiation.cc:299] Negotiation successful
0811 00:26:45.142376 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26920,"thread_start_us":27188,"threads_started":1}
I20260811 00:26:45.143366 10630 heartbeater.cc:511] Master 127.2.120.61:46301 requested a full tablet report, sending...
I20260811 00:26:45.145691 9908 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:45.146122 10222 ts_manager.cc:194] Registered new tserver with Master: 0dcaa36e9c34456da1814e7cc9e27ace (127.2.120.1:38089)
I20260811 00:26:45.239084 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 9774
I20260811 00:26:45.262936 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:45043
--local_ip_for_outbound_sockets=127.2.120.2
--tserver_master_addrs=127.2.120.62:33143
--webserver_port=39953
--webserver_interface=127.2.120.2
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.2.keytab
--principal=oryx/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:45261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.120.62:33143,127.2.120.61:46301
--tserver_master_addrs=127.2.120.62:33143,127.2.120.61:46301,127.2.120.60:32855 with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:26:45.353359 10387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: NON_VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: true } } }
I20260811 00:26:45.353761 10387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:45.355214 10641 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee: attempting to promote NON_VOTER e60785b019bf4273960830aeeb93fd30 to VOTER
I20260811 00:26:45.355906 10641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } }
I20260811 00:26:45.357805 10442 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 1 LEARNER]: Refusing update from remote peer 97e4014ce0a74657ab9899f0ba4c2eee: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 00:26:45.358326 10233 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 1 FOLLOWER]: Refusing update from remote peer 97e4014ce0a74657ab9899f0ba4c2eee: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 00:26:45.358434 10387 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 00:26:45.359040 10641 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 00:26:45.361559 10641 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, e60785b019bf4273960830aeeb93fd30 (127.2.120.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } } }
I20260811 00:26:45.363267 10233 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, e60785b019bf4273960830aeeb93fd30 (127.2.120.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } } }
I20260811 00:26:45.363657 10442 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, e60785b019bf4273960830aeeb93fd30 (127.2.120.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } } }
I20260811 00:26:45.365895 10641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } } }
I20260811 00:26:45.366145 10641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:45.366384 10641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } } }
I20260811 00:26:45.366734 10641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:45.366981 10641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } } }
I20260811 00:26:45.367170 10641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:45.369040 10645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } } }
I20260811 00:26:45.369468 10645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:26:45.370360 10646 sys_catalog.cc:455] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } } }
I20260811 00:26:45.370711 10646 sys_catalog.cc:458] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 00:26:45.671237 10640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:45.671650 10640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:45.671835 10640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:45.679656 10640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:45.679841 10640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:45.680032 10640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:26:45.690145 10640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:45261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.2.keytab
--principal=oryx/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:45043
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=39953
--tserver_master_addrs=127.2.120.62:33143,127.2.120.61:46301,127.2.120.60:32855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:45.691861 10640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:45.693554 10640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:45.707880 10654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:45.707933 10651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:45.707913 10652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:45.709590 10640 server_base.cc:1034] running on GCE node
I20260811 00:26:45.710294 10640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:45.711369 10640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:45.712683 10640 hybrid_clock.cc:648] HybridClock initialized: now 1786408005712627 us; error 52 us; skew 500 ppm
I20260811 00:26:45.713049 10640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:45 dist-test-slave-fd7l krb5kdc[9503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408005, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:45.716714 10640 init.cc:377] Logged in from keytab as oryx/127.2.120.2@KRBTEST.COM (short username oryx)
I20260811 00:26:45.718571 10640 webserver.cc:460] Webserver started at http://127.2.120.2:39953/ using document root <none> and password file <none>
I20260811 00:26:45.719352 10640 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:45.719481 10640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:45.725750 10640 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:26:45.729043 10663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:45.730819 10640 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:26:45.731006 10640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "2150e97860df4cc7bbfbf9b18dc8f86d"
format_stamp: "Formatted at 2026-08-11 00:26:42 on dist-test-slave-fd7l"
server_key: "23f0f8bfe9d83a52b47b1a66cd570548"
server_key_iv: "34eaedd5d01390c0c563a35e559a9c32"
server_key_version: "encryptionkey@0"
I20260811 00:26:45.732574 10640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:45.772588 10640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:45.778198 10640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:45.778579 10640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:45.779793 10640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:45.781497 10640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:45.781600 10640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:45.782065 10640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:45.782167 10640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:45.820294 10640 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:45043
I20260811 00:26:45.820400 10784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:45043 every 8 connection(s)
I20260811 00:26:45.822213 10640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:26:45.824463 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 10640
I20260811 00:26:45.824826 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 9920
Aug 11 00:26:45 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408005, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.60@KRBTEST.COM
Aug 11 00:26:45 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.120.3@KRBTEST.COM
Aug 11 00:26:45 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408005, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.61@KRBTEST.COM
Aug 11 00:26:45 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408005, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM
I20260811 00:26:45.887706 10791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:45.826424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:50631 (local address 127.2.120.60:32855)
0811 00:26:45.859581 (+ 33157us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:45.859591 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:45.859619 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:26:45.859726 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:45.859736 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:45.859896 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:45.860106 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:45.864329 (+ 4223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.866177 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.870300 (+ 4123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.870809 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.871017 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:45.877376 (+ 6359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:45.877432 (+ 56us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:45.877439 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:45.877500 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:45.881599 (+ 4099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:45.882950 (+ 1351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:45.882960 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:45.882984 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:45.883156 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:45.883894 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:45.883901 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:45.883905 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:45.884395 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:45.884575 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:45.886968 (+ 2393us) server_negotiation.cc:299] Negotiation successful
0811 00:26:45.887245 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32974,"thread_start_us":33778,"threads_started":1}
I20260811 00:26:45.888895 10792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:45.826702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:32855 (local address 127.2.120.2:50631)
0811 00:26:45.843230 (+ 16528us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:45.843303 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:45.847681 (+ 4378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:45.860852 (+ 13171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:45.860876 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:45.862338 (+ 1462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:45.863868 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:45.863931 (+ 63us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.866592 (+ 2661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.866604 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:45.870009 (+ 3405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:45.870048 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.871414 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.871424 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:45.871907 (+ 483us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:45.873126 (+ 1219us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:45.873170 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:45.876956 (+ 3786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:45.881805 (+ 4849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:45.881818 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:45.881843 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:45.882732 (+ 889us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:45.883362 (+ 630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:45.883368 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:45.883371 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:45.883708 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:45.884686 (+ 978us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:45.884700 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:45.885279 (+ 579us) client_negotiation.cc:769] Sending connection context
0811 00:26:45.885676 (+ 397us) client_negotiation.cc:240] Negotiation successful
0811 00:26:45.886343 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15203,"mutex_wait_us":42,"thread_start_us":15560,"threads_started":1}
I20260811 00:26:45.891010 10785 heartbeater.cc:347] Connected to a master server at 127.2.120.60:32855
I20260811 00:26:45.891175 10793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:45.826702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:46301 (local address 127.2.120.2:59989)
0811 00:26:45.845027 (+ 18325us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:45.845055 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:45.848320 (+ 3265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:45.867908 (+ 19588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:45.867914 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:45.868722 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:45.869116 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:45.869134 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.871392 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.871399 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:45.873813 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:45.873836 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.875904 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.875913 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:45.876165 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:45.876987 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:45.877017 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:45.880890 (+ 3873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:45.886363 (+ 5473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:45.886368 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:45.886372 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:45.888359 (+ 1987us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:45.888921 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:45.888925 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:45.888929 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:45.889280 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:45.890049 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:45.890054 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:45.890212 (+ 158us) client_negotiation.cc:769] Sending connection context
0811 00:26:45.890345 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 00:26:45.890662 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17752,"mutex_wait_us":149,"spinlock_wait_cycles":17152,"thread_start_us":15368,"threads_started":1}
I20260811 00:26:45.891408 10785 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:45.891456 10794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:45.827109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42256 (local address 127.2.120.3:43095)
0811 00:26:45.834454 (+ 7345us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:45.834470 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:45.861476 (+ 27006us) server_negotiation.cc:407] Connection header received
0811 00:26:45.861949 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:45.861966 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:45.862428 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:45.862613 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:45.866846 (+ 4233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.868754 (+ 1908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.871793 (+ 3039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.872930 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.873069 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:45.882052 (+ 8983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:45.882077 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:45.882101 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:45.882176 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:45.886480 (+ 4304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:45.888048 (+ 1568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:45.888064 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:45.888068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:45.888276 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:45.888956 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:45.888962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:45.888965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:45.889490 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:45.889822 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:45.890634 (+ 812us) server_negotiation.cc:299] Negotiation successful
0811 00:26:45.890938 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7038,"thread_start_us":6791,"threads_started":1}
I20260811 00:26:45.892212 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:45.826067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:59989 (local address 127.2.120.61:46301)
0811 00:26:45.867262 (+ 41195us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:45.867268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:45.867290 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:26:45.867352 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:45.867357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:45.867460 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:45.867675 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:45.870042 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.871231 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.874101 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.874858 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.876640 (+ 1782us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:45.881220 (+ 4580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:45.881229 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:45.881237 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:45.881301 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:45.886167 (+ 4866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:45.888587 (+ 2420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:45.888594 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:45.888598 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:45.888770 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:45.889416 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:45.889421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:45.889424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:45.889817 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:45.889952 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:45.891563 (+ 1611us) server_negotiation.cc:299] Negotiation successful
0811 00:26:45.891820 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41059,"thread_start_us":40892,"threads_started":1}
I20260811 00:26:45.892377 10785 heartbeater.cc:511] Master 127.2.120.60:32855 requested a full tablet report, sending...
I20260811 00:26:45.892561 10796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:45.844642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:37695 (local address 127.2.120.62:33143)
0811 00:26:45.857527 (+ 12885us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:45.857539 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:45.866703 (+ 9164us) server_negotiation.cc:407] Connection header received
0811 00:26:45.871262 (+ 4559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:45.871269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:45.871391 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:45.871548 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:45.873130 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.874376 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.876793 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.877589 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.877712 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:45.883098 (+ 5386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:45.883106 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:45.883112 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:45.883180 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:45.886815 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:45.887771 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:45.887778 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:45.887781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:45.887960 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:45.888594 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:45.888599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:45.888601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:45.889030 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:45.889201 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:45.890267 (+ 1066us) server_negotiation.cc:299] Negotiation successful
0811 00:26:45.890558 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12668,"thread_start_us":12567,"threads_started":1}
I20260811 00:26:45.893599 10795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:45.843972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.2:37695)
0811 00:26:45.866575 (+ 22603us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:45.866600 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:45.869819 (+ 3219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:45.871601 (+ 1782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:45.871606 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:45.872451 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:45.872872 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:45.872889 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.874604 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.874611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:45.876566 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:45.876589 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:45.878081 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:45.878089 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:45.878357 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:45.879042 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:45.879068 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:45.882827 (+ 3759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:45.887004 (+ 4177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:45.887008 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:45.887011 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:45.887604 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:45.888236 (+ 632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:45.888239 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:45.888242 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:45.888393 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:45.889280 (+ 887us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:45.889285 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:45.889446 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 00:26:45.892827 (+ 3381us) client_negotiation.cc:240] Negotiation successful
0811 00:26:45.893152 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22434,"thread_start_us":22244,"threads_started":1}
I20260811 00:26:45.893774 10787 heartbeater.cc:347] Connected to a master server at 127.2.120.61:46301
I20260811 00:26:45.893904 10787 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:45.894035 10057 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:45.894434 10787 heartbeater.cc:511] Master 127.2.120.61:46301 requested a full tablet report, sending...
I20260811 00:26:45.894881 10786 heartbeater.cc:347] Connected to a master server at 127.2.120.62:33143
I20260811 00:26:45.895022 10786 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:45.894981 10432 ts_manager.cc:194] Registered new tserver with Master: 2150e97860df4cc7bbfbf9b18dc8f86d (127.2.120.2:45043)
I20260811 00:26:45.895447 10786 heartbeater.cc:511] Master 127.2.120.62:33143 requested a full tablet report, sending...
I20260811 00:26:45.896045 10222 ts_manager.cc:194] Registered new tserver with Master: 2150e97860df4cc7bbfbf9b18dc8f86d (127.2.120.2:45043)
I20260811 00:26:45.896929 9554 ts_manager.cc:194] Re-registered known tserver with Master: 2150e97860df4cc7bbfbf9b18dc8f86d (127.2.120.2:45043)
I20260811 00:26:45.898310 9554 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:37695
I20260811 00:26:45.989514 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 9920
I20260811 00:26:46.009774 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:43095
--local_ip_for_outbound_sockets=127.2.120.3
--tserver_master_addrs=127.2.120.62:33143
--webserver_port=38883
--webserver_interface=127.2.120.3
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.3.keytab
--principal=oryx/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:45261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.120.62:33143,127.2.120.61:46301
--tserver_master_addrs=127.2.120.62:33143,127.2.120.61:46301,127.2.120.60:32855 with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:26:46.146844 10629 heartbeater.cc:503] Master 127.2.120.62:33143 was elected leader, sending a full tablet report...
W20260811 00:26:46.410413 10798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:46.410758 10798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:46.410933 10798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:46.419003 10798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:46.419219 10798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:46.419422 10798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:26:46.430119 10798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:45261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.3.keytab
--principal=oryx/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:43095
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=38883
--tserver_master_addrs=127.2.120.62:33143,127.2.120.61:46301,127.2.120.60:32855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:46.431625 10798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:46.433139 10798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:46.448951 10804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:46.449044 10807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:46.449044 10803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:46.449585 10798 server_base.cc:1034] running on GCE node
I20260811 00:26:46.450410 10798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:46.451628 10798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:46.452869 10798 hybrid_clock.cc:648] HybridClock initialized: now 1786408006452821 us; error 47 us; skew 500 ppm
I20260811 00:26:46.453230 10798 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:46 dist-test-slave-fd7l krb5kdc[9503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408006, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:46.457086 10798 init.cc:377] Logged in from keytab as oryx/127.2.120.3@KRBTEST.COM (short username oryx)
I20260811 00:26:46.458894 10798 webserver.cc:460] Webserver started at http://127.2.120.3:38883/ using document root <none> and password file <none>
I20260811 00:26:46.459684 10798 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:46.459913 10798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:46.466842 10798 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 00:26:46.470587 10815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:46.472208 10798 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:26:46.472425 10798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "e0691687a81d4375ac1258533cdce5eb"
format_stamp: "Formatted at 2026-08-11 00:26:42 on dist-test-slave-fd7l"
server_key: "26c3fcc1af1c3cde9fcd7bef8fc9eb5c"
server_key_iv: "12b62ccd3bc24b0f97443a340f1b8d25"
server_key_version: "encryptionkey@0"
I20260811 00:26:46.473589 10798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:46.489589 10798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:46.495507 10798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:46.495904 10798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:46.497211 10798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:26:46.499032 10798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:46.499181 10798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:46.499334 10798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:46.499434 10798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:46.540364 10798 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:43095
I20260811 00:26:46.540447 10940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:43095 every 8 connection(s)
I20260811 00:26:46.542192 10798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:26:46.542829 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 10798
Aug 11 00:26:46 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408006, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM
Aug 11 00:26:46 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408006, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.61@KRBTEST.COM
Aug 11 00:26:46 dist-test-slave-fd7l krb5kdc[9503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408006, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.60@KRBTEST.COM
I20260811 00:26:46.614743 10949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.547212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:46301 (local address 127.2.120.3:37365)
0811 00:26:46.565370 (+ 18158us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:46.565434 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:46.569360 (+ 3926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:46.571293 (+ 1933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:46.571315 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:46.572796 (+ 1481us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:46.574199 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:46.574241 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.575592 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.575598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:46.578393 (+ 2795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:46.578414 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.579856 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.579866 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:46.580204 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.587232 (+ 7028us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:46.587270 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:46.593000 (+ 5730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:46.607324 (+ 14324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:46.607338 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:46.607362 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:46.608182 (+ 820us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:46.608846 (+ 664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:46.608851 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:46.608855 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:46.609145 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:46.611525 (+ 2380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:46.611535 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:46.612076 (+ 541us) client_negotiation.cc:769] Sending connection context
0811 00:26:46.612388 (+ 312us) client_negotiation.cc:240] Negotiation successful
0811 00:26:46.613050 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17740,"mutex_wait_us":117,"spinlock_wait_cycles":47104,"thread_start_us":26666,"threads_started":1}
I20260811 00:26:46.616701 10950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.548141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:37365 (local address 127.2.120.61:46301)
0811 00:26:46.570056 (+ 21915us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:46.570065 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:46.570099 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:26:46.570210 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:46.570219 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:46.570360 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:46.570758 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:46.574578 (+ 3820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.575387 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.578657 (+ 3270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.579348 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.579482 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.593303 (+ 13821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:46.593311 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:46.593315 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:46.593366 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:46.607110 (+ 13744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:46.608397 (+ 1287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:46.608403 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:46.608408 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:46.608705 (+ 297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:46.609302 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:46.609308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:46.609316 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:46.609714 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:46.615691 (+ 5977us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:46.615883 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 00:26:46.616229 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21691,"thread_start_us":22802,"threads_started":1}
I20260811 00:26:46.618415 10943 heartbeater.cc:347] Connected to a master server at 127.2.120.61:46301
I20260811 00:26:46.618854 10943 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:46.620028 10943 heartbeater.cc:511] Master 127.2.120.61:46301 requested a full tablet report, sending...
I20260811 00:26:46.622058 10951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.566490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.3:49455)
0811 00:26:46.574903 (+ 8413us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:46.574935 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:46.575075 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:46.577090 (+ 2015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:46.577099 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:46.578070 (+ 971us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:46.578644 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:46.578662 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.581605 (+ 2943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.581609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:46.582647 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:46.582661 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.583591 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.583598 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:46.583794 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.587200 (+ 3406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:46.587270 (+ 70us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:46.592547 (+ 5277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:46.615578 (+ 23031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:46.615585 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:46.615591 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:46.616398 (+ 807us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:46.617792 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:46.617798 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:46.617802 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:46.618028 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:46.620791 (+ 2763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:46.620798 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:46.620979 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 00:26:46.621184 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 00:26:46.621556 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8244,"mutex_wait_us":2,"spinlock_wait_cycles":20096,"thread_start_us":8059,"threads_started":1}
I20260811 00:26:46.622704 10222 ts_manager.cc:194] Registered new tserver with Master: e0691687a81d4375ac1258533cdce5eb (127.2.120.3:43095)
I20260811 00:26:46.624461 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.566661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:49455 (local address 127.2.120.62:33143)
0811 00:26:46.576446 (+ 9785us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:46.576458 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:46.576494 (+ 36us) server_negotiation.cc:407] Connection header received
0811 00:26:46.576612 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:46.576621 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:46.576773 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:46.578897 (+ 2124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:46.579163 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.581427 (+ 2264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.582833 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.583377 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.583534 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.593377 (+ 9843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:46.593384 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:46.593390 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:46.593444 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:46.615404 (+ 21960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:46.617444 (+ 2040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:46.617450 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:46.617453 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:46.617637 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:46.618631 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:46.618637 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:46.618640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:46.620632 (+ 1992us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:46.622536 (+ 1904us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:46.622680 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 00:26:46.624054 (+ 1374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9430,"thread_start_us":9175,"threads_started":1}
I20260811 00:26:46.624718 10948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.547804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:32855 (local address 127.2.120.3:34317)
0811 00:26:46.571778 (+ 23974us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:46.571820 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:46.572102 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:46.573047 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:46.573055 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:46.573906 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:46.575847 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:46.575863 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.577457 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.577462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:46.579084 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:46.579098 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.580409 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.580416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:46.580672 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.589782 (+ 9110us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:46.589822 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:46.600261 (+ 10439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:46.609880 (+ 9619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:46.609885 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:46.609890 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:46.610935 (+ 1045us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:46.616683 (+ 5748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:46.616690 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:46.616695 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:46.616893 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:46.623535 (+ 6642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:46.623540 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:46.623731 (+ 191us) client_negotiation.cc:769] Sending connection context
0811 00:26:46.623924 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 00:26:46.624256 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23736,"mutex_wait_us":1464,"thread_start_us":16985,"threads_started":1}
I20260811 00:26:46.625551 10956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.583184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43772 (local address 127.2.120.62:33143)
0811 00:26:46.595304 (+ 12120us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:46.595315 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:46.595346 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:26:46.595431 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:46.595439 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:46.595558 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:46.603589 (+ 8031us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:46.603903 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.605581 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.608242 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.609243 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.609398 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.614251 (+ 4853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:46.614262 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:46.614267 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:46.614323 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:46.620409 (+ 6086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:46.621408 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:46.621414 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:46.621417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:46.621578 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:46.622077 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:46.622085 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:46.622089 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:46.622477 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:46.623839 (+ 1362us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:46.623991 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 00:26:46.624279 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11907,"thread_start_us":11729,"threads_started":1}
I20260811 00:26:46.625721 10942 heartbeater.cc:347] Connected to a master server at 127.2.120.62:33143
I20260811 00:26:46.625882 10942 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:46.626427 10942 heartbeater.cc:511] Master 127.2.120.62:33143 requested a full tablet report, sending...
I20260811 00:26:46.626650 10947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.547496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:34317 (local address 127.2.120.60:32855)
0811 00:26:46.570429 (+ 22933us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:46.570439 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:46.572344 (+ 1905us) server_negotiation.cc:407] Connection header received
0811 00:26:46.572446 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:46.572455 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:46.572620 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:46.572773 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:46.576072 (+ 3299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.577234 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.579282 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.579961 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.580069 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.600909 (+ 20840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:46.600921 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:46.600930 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:46.601001 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:46.606514 (+ 5513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:46.611157 (+ 4643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:46.611163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:46.611167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:46.611351 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:46.617092 (+ 5741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:46.617098 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:46.617102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:46.623363 (+ 6261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:46.625400 (+ 2037us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:46.625586 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 00:26:46.626203 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22752,"thread_start_us":22629,"threads_started":1}
I20260811 00:26:46.627542 9554 ts_manager.cc:194] Re-registered known tserver with Master: e0691687a81d4375ac1258533cdce5eb (127.2.120.3:43095)
I20260811 00:26:46.628091 10941 heartbeater.cc:347] Connected to a master server at 127.2.120.60:32855
I20260811 00:26:46.628258 10941 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:46.628942 10941 heartbeater.cc:511] Master 127.2.120.60:32855 requested a full tablet report, sending...
I20260811 00:26:46.628925 9554 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:49455
I20260811 00:26:46.631611 10432 ts_manager.cc:194] Registered new tserver with Master: e0691687a81d4375ac1258533cdce5eb (127.2.120.3:43095)
I20260811 00:26:46.633890 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.584785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49032 (local address 127.2.120.61:46301)
0811 00:26:46.598384 (+ 13599us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:46.598396 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:46.598443 (+ 47us) server_negotiation.cc:407] Connection header received
0811 00:26:46.598565 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:46.598573 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:46.598703 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:46.599109 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:46.609205 (+ 10096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.610725 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.613417 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.614332 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.614485 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.621850 (+ 7365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:46.621858 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:46.621862 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:46.621912 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:46.626321 (+ 4409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:46.627702 (+ 1381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:46.627710 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:46.627713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:46.628920 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:46.629879 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:46.629886 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:46.629890 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:46.630760 (+ 870us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:46.631077 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:46.632915 (+ 1838us) server_negotiation.cc:299] Negotiation successful
0811 00:26:46.633307 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":417,"threads_started":1}
I20260811 00:26:46.635179 9554 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 00:26:46.647357 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.585511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36670 (local address 127.2.120.60:32855)
0811 00:26:46.605507 (+ 19996us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:46.605514 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:46.606334 (+ 820us) server_negotiation.cc:407] Connection header received
0811 00:26:46.606713 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:46.606720 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:46.606808 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:46.606935 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:46.616940 (+ 10005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.618291 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.624604 (+ 6313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.625339 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.625454 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.635778 (+ 10324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:46.635792 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:46.635799 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:46.635892 (+ 93us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:46.641806 (+ 5914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:46.643600 (+ 1794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:46.643609 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:46.643612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:46.643983 (+ 371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:46.644782 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:46.644788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:46.644792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:46.645278 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:46.645679 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:46.646322 (+ 643us) server_negotiation.cc:299] Negotiation successful
0811 00:26:46.646780 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19814,"spinlock_wait_cycles":1792,"thread_start_us":22919,"threads_started":1}
W20260811 00:26:46.649497 9554 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:26:46.691557 10974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.676899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:38089 (local address 127.0.0.1:41840)
0811 00:26:46.678655 (+ 1756us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:46.678685 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:46.678881 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:46.679888 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:46.679896 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:46.679937 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:46.680567 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:46.680588 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.683499 (+ 2911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.683510 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:46.686836 (+ 3326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:46.686873 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.690533 (+ 3660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.690541 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:46.690772 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.690819 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 00:26:46.690902 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 00:26:46.691125 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1525,"thread_start_us":1151,"threads_started":1}
I20260811 00:26:46.691916 10975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.677327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41840 (local address 127.2.120.1:38089)
0811 00:26:46.678798 (+ 1471us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:46.678812 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:46.678861 (+ 49us) server_negotiation.cc:407] Connection header received
0811 00:26:46.679154 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:46.679167 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:46.679574 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:46.679878 (+ 304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:46.680871 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.683160 (+ 2289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.687159 (+ 3999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.690232 (+ 3073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.690465 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.690863 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:46.690976 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 00:26:46.691284 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1276,"thread_start_us":1128,"threads_started":1}
I20260811 00:26:46.698508 10559 tablet_service.cc:1467] Processing CreateTablet for tablet 3d683dff276c435d86b291ab29b6ca1c (DEFAULT_TABLE table=test-table [id=dd0100815c9343d3bfc01209548db46a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:46.700438 10976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.677993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:45043 (local address 127.0.0.1:48958)
0811 00:26:46.684051 (+ 6058us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:46.684080 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:46.684257 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:46.690153 (+ 5896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:46.690160 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:46.690198 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:46.690647 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:46.690662 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.692612 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.692618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:46.695769 (+ 3151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:46.695801 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.699154 (+ 3353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.699164 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:46.699466 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.699528 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 00:26:46.699680 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 00:26:46.699865 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5844,"thread_start_us":5478,"threads_started":1}
I20260811 00:26:46.701175 10559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d683dff276c435d86b291ab29b6ca1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:46.701529 10978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.678262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48958 (local address 127.2.120.2:45043)
0811 00:26:46.689473 (+ 11211us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:46.689481 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:46.689514 (+ 33us) server_negotiation.cc:407] Connection header received
0811 00:26:46.689617 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:46.689624 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:46.689867 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:46.690149 (+ 282us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:46.690872 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.692407 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.696194 (+ 3787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.698374 (+ 2180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.698620 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.698920 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:46.700982 (+ 2062us) server_negotiation.cc:299] Negotiation successful
0811 00:26:46.701207 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11014,"thread_start_us":10849,"threads_started":1}
I20260811 00:26:46.705827 10717 tablet_service.cc:1467] Processing CreateTablet for tablet 3d683dff276c435d86b291ab29b6ca1c (DEFAULT_TABLE table=test-table [id=dd0100815c9343d3bfc01209548db46a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:46.707551 10717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d683dff276c435d86b291ab29b6ca1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:46.716953 10974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.684774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:43095 (local address 127.0.0.1:42258)
0811 00:26:46.692344 (+ 7570us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:46.692365 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:46.692504 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:46.703254 (+ 10750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:46.703259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:46.703295 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:46.703655 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:46.703673 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.706513 (+ 2840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.706519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:46.708422 (+ 1903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:46.708454 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.715879 (+ 7425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.715887 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:46.716175 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.716232 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 00:26:46.716318 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 00:26:46.716505 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7446,"thread_start_us":16081,"threads_started":1}
I20260811 00:26:46.718539 10979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.684385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42258 (local address 127.2.120.3:43095)
0811 00:26:46.702419 (+ 18034us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:46.702429 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:46.702469 (+ 40us) server_negotiation.cc:407] Connection header received
0811 00:26:46.702590 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:46.702603 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:46.702902 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:46.703249 (+ 347us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:46.703849 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.706304 (+ 2455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.713209 (+ 6905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.715052 (+ 1843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.717455 (+ 2403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.717778 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:46.717875 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 00:26:46.718095 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17815,"thread_start_us":19159,"threads_started":1}
I20260811 00:26:46.724620 10869 tablet_service.cc:1467] Processing CreateTablet for tablet 3d683dff276c435d86b291ab29b6ca1c (DEFAULT_TABLE table=test-table [id=dd0100815c9343d3bfc01209548db46a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:46.727361 10869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d683dff276c435d86b291ab29b6ca1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:46.741504 10981 tablet_bootstrap.cc:492] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace: Bootstrap starting.
I20260811 00:26:46.742692 10982 tablet_bootstrap.cc:492] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d: Bootstrap starting.
I20260811 00:26:46.747164 10982 tablet_bootstrap.cc:654] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:46.747148 10981 tablet_bootstrap.cc:654] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:46.748459 10982 log.cc:824] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:46.748625 10981 log.cc:824] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:46.752058 10982 tablet_bootstrap.cc:492] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d: No bootstrap required, opened a new log
I20260811 00:26:46.752418 10982 ts_tablet_manager.cc:1397] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.005s
I20260811 00:26:46.753760 10985 tablet_bootstrap.cc:492] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb: Bootstrap starting.
I20260811 00:26:46.754961 10981 tablet_bootstrap.cc:492] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace: No bootstrap required, opened a new log
I20260811 00:26:46.755388 10981 ts_tablet_manager.cc:1397] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.004s
I20260811 00:26:46.757678 10985 tablet_bootstrap.cc:654] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:46.758927 10985 log.cc:824] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:46.762725 10985 tablet_bootstrap.cc:492] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb: No bootstrap required, opened a new log
I20260811 00:26:46.763087 10985 ts_tablet_manager.cc:1397] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260811 00:26:46.766669 10982 raft_consensus.cc:348] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } }
I20260811 00:26:46.767241 10982 raft_consensus.cc:374] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:46.767390 10982 raft_consensus.cc:729] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2150e97860df4cc7bbfbf9b18dc8f86d, State: Initialized, Role: FOLLOWER
I20260811 00:26:46.768183 10982 consensus_queue.cc:260] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } }
I20260811 00:26:46.768314 10981 raft_consensus.cc:348] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } }
I20260811 00:26:46.768851 10981 raft_consensus.cc:374] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:46.769069 10981 raft_consensus.cc:729] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dcaa36e9c34456da1814e7cc9e27ace, State: Initialized, Role: FOLLOWER
I20260811 00:26:46.769629 10786 heartbeater.cc:503] Master 127.2.120.62:33143 was elected leader, sending a full tablet report...
I20260811 00:26:46.770380 10982 ts_tablet_manager.cc:1428] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d: Time spent starting tablet: real 0.018s user 0.013s sys 0.002s
I20260811 00:26:46.772480 10985 raft_consensus.cc:348] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } }
I20260811 00:26:46.773496 10985 raft_consensus.cc:374] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:46.773272 10981 consensus_queue.cc:260] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } }
I20260811 00:26:46.773693 10985 raft_consensus.cc:729] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0691687a81d4375ac1258533cdce5eb, State: Initialized, Role: FOLLOWER
I20260811 00:26:46.774634 10985 consensus_queue.cc:260] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } }
I20260811 00:26:46.775655 10981 ts_tablet_manager.cc:1428] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace: Time spent starting tablet: real 0.020s user 0.015s sys 0.000s
I20260811 00:26:46.780416 10942 heartbeater.cc:503] Master 127.2.120.62:33143 was elected leader, sending a full tablet report...
I20260811 00:26:46.783002 10985 ts_tablet_manager.cc:1428] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb: Time spent starting tablet: real 0.020s user 0.012s sys 0.006s
W20260811 00:26:46.813212 10946 tablet.cc:2369] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:26:46.827438 10788 tablet.cc:2369] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:26:46.852583 10637 tablet.cc:2369] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:26:46.886905 10988 raft_consensus.cc:482] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:26:46.887251 10988 raft_consensus.cc:504] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } }
I20260811 00:26:46.889094 10988 leader_election.cc:288] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2150e97860df4cc7bbfbf9b18dc8f86d (127.2.120.2:45043), e0691687a81d4375ac1258533cdce5eb (127.2.120.3:43095)
I20260811 00:26:46.897038 10978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.889628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:53341 (local address 127.2.120.2:45043)
0811 00:26:46.889831 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:46.889836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:46.890336 (+ 500us) server_negotiation.cc:407] Connection header received
0811 00:26:46.890751 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:46.890760 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:46.890855 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:46.890921 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:46.891640 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.892708 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.894971 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.895846 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.896024 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.896160 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:46.896592 (+ 432us) server_negotiation.cc:299] Negotiation successful
0811 00:26:46.896730 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:26:46.897163 10979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.890699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:52587 (local address 127.2.120.3:43095)
0811 00:26:46.890971 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:46.890977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:46.891198 (+ 221us) server_negotiation.cc:407] Connection header received
0811 00:26:46.891437 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:46.891443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:46.891516 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:46.891644 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:46.892123 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.893118 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.895156 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.896094 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.896264 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.896383 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:46.896757 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 00:26:46.896900 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 00:26:46.897267 10990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.889471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:45043 (local address 127.2.120.1:53341)
0811 00:26:46.890220 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:46.890248 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:46.890496 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:46.891133 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:46.891136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:46.891157 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:46.891478 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:46.891487 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.892871 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.892875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:46.894767 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:46.894785 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.896155 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.896162 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:46.896366 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.896394 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 00:26:46.896519 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 00:26:46.896697 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":25,"thread_start_us":367,"threads_started":1}
I20260811 00:26:46.897318 10991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.890313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:43095 (local address 127.2.120.1:52587)
0811 00:26:46.891083 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:46.891097 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:46.891283 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:46.891653 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:46.891656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:46.891674 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:46.891950 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:46.891957 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.893277 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.893289 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:46.894967 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:46.894982 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.896358 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.896367 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:46.896592 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.896618 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 00:26:46.896727 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 00:26:46.896867 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":42,"thread_start_us":487,"threads_started":1}
I20260811 00:26:46.898823 10739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d683dff276c435d86b291ab29b6ca1c" candidate_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" is_pre_election: true
I20260811 00:26:46.898823 10891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d683dff276c435d86b291ab29b6ca1c" candidate_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0691687a81d4375ac1258533cdce5eb" is_pre_election: true
I20260811 00:26:46.899485 10891 raft_consensus.cc:2454] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dcaa36e9c34456da1814e7cc9e27ace in term 0.
I20260811 00:26:46.899483 10739 raft_consensus.cc:2454] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dcaa36e9c34456da1814e7cc9e27ace in term 0.
I20260811 00:26:46.900367 10508 leader_election.cc:302] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dcaa36e9c34456da1814e7cc9e27ace, 2150e97860df4cc7bbfbf9b18dc8f86d; no voters:
I20260811 00:26:46.900971 10988 raft_consensus.cc:2781] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:26:46.901190 10988 raft_consensus.cc:482] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:26:46.901329 10988 raft_consensus.cc:3037] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:46.905110 10988 raft_consensus.cc:504] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } }
I20260811 00:26:46.905937 10988 leader_election.cc:288] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [CANDIDATE]: Term 1 election: Requested vote from peers 2150e97860df4cc7bbfbf9b18dc8f86d (127.2.120.2:45043), e0691687a81d4375ac1258533cdce5eb (127.2.120.3:43095)
I20260811 00:26:46.906714 10739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d683dff276c435d86b291ab29b6ca1c" candidate_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d"
I20260811 00:26:46.906793 10891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d683dff276c435d86b291ab29b6ca1c" candidate_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0691687a81d4375ac1258533cdce5eb"
I20260811 00:26:46.906993 10739 raft_consensus.cc:3037] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:46.907027 10891 raft_consensus.cc:3037] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:46.909690 10739 raft_consensus.cc:2454] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dcaa36e9c34456da1814e7cc9e27ace in term 1.
I20260811 00:26:46.910336 10508 leader_election.cc:302] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dcaa36e9c34456da1814e7cc9e27ace, 2150e97860df4cc7bbfbf9b18dc8f86d; no voters:
I20260811 00:26:46.910598 10891 raft_consensus.cc:2454] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dcaa36e9c34456da1814e7cc9e27ace in term 1.
I20260811 00:26:46.910772 10988 raft_consensus.cc:2781] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:46.911598 10988 raft_consensus.cc:686] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [term 1 LEADER]: Becoming Leader. State: Replica: 0dcaa36e9c34456da1814e7cc9e27ace, State: Running, Role: LEADER
I20260811 00:26:46.912489 10988 consensus_queue.cc:237] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } }
I20260811 00:26:46.918740 9553 catalog_manager.cc:5495] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace reported cstate change: term changed from 0 to 1, leader changed from <none> to 0dcaa36e9c34456da1814e7cc9e27ace (127.2.120.1). New cstate: current_term: 1 leader_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } health_report { overall_health: UNKNOWN } } }
I20260811 00:26:46.962322 10975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:46.956193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41856 (local address 127.2.120.1:38089)
0811 00:26:46.956422 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:46.956427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:46.956447 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:26:46.956593 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:46.956597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:46.956686 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:46.956831 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:26:46.957382 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.958216 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.959734 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:46.960189 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:46.960353 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:46.961064 (+ 711us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:26:46.961636 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:26:46.961808 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:46.961909 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 00:26:46.962001 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 00:26:46.972446 10739 raft_consensus.cc:1261] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d [term 1 FOLLOWER]: Refusing update from remote peer 0dcaa36e9c34456da1814e7cc9e27ace: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:46.972451 10891 raft_consensus.cc:1261] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb [term 1 FOLLOWER]: Refusing update from remote peer 0dcaa36e9c34456da1814e7cc9e27ace: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:46.973769 10994 consensus_queue.cc:1035] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:46.974267 10988 consensus_queue.cc:1035] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [LEADER]: Connected to new peer: Peer: permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:47.014034 9554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:47.014384 9554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:47.023311 9554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97e4014ce0a74657ab9899f0ba4c2eee: Sending DeleteTablet for 3 replicas of tablet 3d683dff276c435d86b291ab29b6ca1c
I20260811 00:26:47.024663 10559 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d683dff276c435d86b291ab29b6ca1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:47 UTC) from {username='oryx', principal='oryx/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:41840
I20260811 00:26:47.025030 10717 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d683dff276c435d86b291ab29b6ca1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:47 UTC) from {username='oryx', principal='oryx/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:48958
I20260811 00:26:47.025689 10869 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d683dff276c435d86b291ab29b6ca1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:47 UTC) from {username='oryx', principal='oryx/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:42258
I20260811 00:26:47.037633 11010 tablet_replica.cc:330] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace: stopping tablet replica
I20260811 00:26:47.039841 11010 raft_consensus.cc:2229] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:26:47.041049 11013 tablet_replica.cc:330] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb: stopping tablet replica
I20260811 00:26:47.041246 11010 raft_consensus.cc:2258] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:47.041795 11013 raft_consensus.cc:2229] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:47.042433 11013 raft_consensus.cc:2258] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:47.043149 11011 tablet_replica.cc:330] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d: stopping tablet replica
I20260811 00:26:47.043637 11011 raft_consensus.cc:2229] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:47.044207 11011 raft_consensus.cc:2258] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:47.045243 11013 ts_tablet_manager.cc:1905] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:47.045603 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 9519
I20260811 00:26:47.047106 9586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:47.053632 11013 ts_tablet_manager.cc:1918] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:26:47.054613 11013 log.cc:1196] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/3d683dff276c435d86b291ab29b6ca1c
I20260811 00:26:47.054842 11010 ts_tablet_manager.cc:1905] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:47.055699 11013 ts_tablet_manager.cc:1939] T 3d683dff276c435d86b291ab29b6ca1c P e0691687a81d4375ac1258533cdce5eb: Deleting consensus metadata
I20260811 00:26:47.058149 11011 ts_tablet_manager.cc:1905] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:47.069092 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:47.028611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49040 (local address 127.2.120.61:46301)
0811 00:26:47.028884 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:47.028893 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:47.045403 (+ 16510us) server_negotiation.cc:407] Connection header received
0811 00:26:47.047171 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:47.047177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:47.047274 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:47.047357 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:47.049897 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:47.051055 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:47.053256 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:47.053970 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:47.054103 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:47.059928 (+ 5825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:47.059937 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:47.059944 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:47.060012 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:47.064798 (+ 4786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:47.066279 (+ 1481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:47.066289 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:47.066296 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:47.066545 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:47.067072 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:47.067078 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:47.067081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:47.067461 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:47.067732 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:47.068383 (+ 651us) server_negotiation.cc:299] Negotiation successful
0811 00:26:47.068714 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 00:26:47.070740 11010 ts_tablet_manager.cc:1918] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:26:47.074677 11011 ts_tablet_manager.cc:1918] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:26:47.075032 11010 log.cc:1196] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/3d683dff276c435d86b291ab29b6ca1c
I20260811 00:26:47.075759 11010 ts_tablet_manager.cc:1939] T 3d683dff276c435d86b291ab29b6ca1c P 0dcaa36e9c34456da1814e7cc9e27ace: Deleting consensus metadata
I20260811 00:26:47.078014 11011 log.cc:1196] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/3d683dff276c435d86b291ab29b6ca1c
I20260811 00:26:47.078859 11011 ts_tablet_manager.cc:1939] T 3d683dff276c435d86b291ab29b6ca1c P 2150e97860df4cc7bbfbf9b18dc8f86d: Deleting consensus metadata
I20260811 00:26:47.090327 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:47.044151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.2.120.60:32855)
0811 00:26:47.044379 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:47.044385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:47.070564 (+ 26179us) server_negotiation.cc:407] Connection header received
0811 00:26:47.070838 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:47.070844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:47.070928 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:47.071004 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:47.073071 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:47.074226 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:47.076276 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:47.076968 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:47.077102 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:47.081342 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:47.081349 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:47.081353 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:47.081401 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:47.086269 (+ 4868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:47.087462 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:47.087470 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:47.087475 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:47.087668 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:47.088181 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:47.088186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:47.088188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:47.088658 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:47.088953 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:47.089673 (+ 720us) server_negotiation.cc:299] Negotiation successful
0811 00:26:47.089965 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 00:26:47.197183 9540 catalog_manager.cc:4841] TS 0dcaa36e9c34456da1814e7cc9e27ace (127.2.120.1:38089): tablet 3d683dff276c435d86b291ab29b6ca1c (table test-table [id=dd0100815c9343d3bfc01209548db46a]) successfully deleted
I20260811 00:26:47.197768 9541 catalog_manager.cc:4841] TS 2150e97860df4cc7bbfbf9b18dc8f86d (127.2.120.2:45043): tablet 3d683dff276c435d86b291ab29b6ca1c (table test-table [id=dd0100815c9343d3bfc01209548db46a]) successfully deleted
I20260811 00:26:47.197973 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 9519
I20260811 00:26:47.261273 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:47.238521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49048 (local address 127.2.120.61:46301)
0811 00:26:47.238743 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:47.238750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:47.239803 (+ 1053us) server_negotiation.cc:407] Connection header received
0811 00:26:47.240021 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:47.240031 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:47.240142 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:47.240268 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:47.245554 (+ 5286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:47.247145 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:47.249572 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:47.250249 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:47.250414 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:47.253299 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:47.253306 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:47.253311 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:47.253358 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:47.257067 (+ 3709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:47.258285 (+ 1218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:47.258292 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:47.258296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:47.258481 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:47.258991 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:47.258997 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:47.259000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:47.259433 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:47.259791 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:47.260473 (+ 682us) server_negotiation.cc:299] Negotiation successful
0811 00:26:47.260867 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 00:26:47.261313 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:47.240679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36692 (local address 127.2.120.60:32855)
0811 00:26:47.240908 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:47.240918 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:47.240966 (+ 48us) server_negotiation.cc:407] Connection header received
0811 00:26:47.241186 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:47.241193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:47.241280 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:47.241353 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:47.244166 (+ 2813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:47.245503 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:47.248273 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:47.249009 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:47.249166 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:47.253231 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:47.253239 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:47.253244 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:47.253309 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:47.257024 (+ 3715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:47.258234 (+ 1210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:47.258241 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:47.258245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:47.258423 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:47.258902 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:47.258908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:47.258912 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:47.259404 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:47.259671 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:47.260414 (+ 743us) server_negotiation.cc:299] Negotiation successful
0811 00:26:47.260758 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 00:26:47.999352 11023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:47.998600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.2:43043)
0811 00:26:47.999052 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:47.999177 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":219,"threads_started":1}
W20260811 00:26:47.999897 10786 heartbeater.cc:644] Failed to heartbeat to 127.2.120.62:33143 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
I20260811 00:26:48.001655 11024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:48.000651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.3:48915)
0811 00:26:48.001236 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:48.001420 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":308,"threads_started":1}
W20260811 00:26:48.002168 10942 heartbeater.cc:644] Failed to heartbeat to 127.2.120.62:33143 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
I20260811 00:26:48.007877 11025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:48.006532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.1:58471)
0811 00:26:48.007307 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:48.007555 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":378,"threads_started":1}
W20260811 00:26:48.008738 10629 heartbeater.cc:644] Failed to heartbeat to 127.2.120.62:33143 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
I20260811 00:26:48.522778 11026 raft_consensus.cc:482] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 97e4014ce0a74657ab9899f0ba4c2eee)
I20260811 00:26:48.523139 11028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 97e4014ce0a74657ab9899f0ba4c2eee)
I20260811 00:26:48.523180 11026 raft_consensus.cc:504] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } }
I20260811 00:26:48.523356 11028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } }
I20260811 00:26:48.524350 11028 leader_election.cc:288] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97e4014ce0a74657ab9899f0ba4c2eee (127.2.120.62:33143), e60785b019bf4273960830aeeb93fd30 (127.2.120.60:32855)
I20260811 00:26:48.530085 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:48.524714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:44206)
0811 00:26:48.529440 (+ 4726us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:48.529706 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":292,"threads_started":1}
I20260811 00:26:48.531313 11026 leader_election.cc:288] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97e4014ce0a74657ab9899f0ba4c2eee (127.2.120.62:33143), 5f58553db3fb46c49fa58a3ea3838cba (127.2.120.61:46301)
I20260811 00:26:48.532682 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:48.525383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:44214)
0811 00:26:48.532012 (+ 6629us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:48.532312 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6365,"thread_start_us":6670,"threads_started":1}
W20260811 00:26:48.533200 10414 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
W20260811 00:26:48.534364 10205 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
I20260811 00:26:48.538431 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:48.537653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:44226)
0811 00:26:48.538008 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:48.538135 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
W20260811 00:26:48.538887 10414 leader_election.cc:334] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 97e4014ce0a74657ab9899f0ba4c2eee (127.2.120.62:33143): Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
I20260811 00:26:48.539688 11030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:48.538551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:44240)
0811 00:26:48.538972 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:48.539254 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228}
W20260811 00:26:48.540096 10205 leader_election.cc:334] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 97e4014ce0a74657ab9899f0ba4c2eee (127.2.120.62:33143): Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
I20260811 00:26:48.544621 11034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:48.525964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36700 (local address 127.2.120.60:32855)
0811 00:26:48.535340 (+ 9376us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:48.535350 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:48.535380 (+ 30us) server_negotiation.cc:407] Connection header received
0811 00:26:48.535485 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:48.535493 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:48.535643 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:48.536342 (+ 699us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:48.537458 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:48.539050 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:48.541866 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:48.543281 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:48.543518 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:48.543663 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:48.543979 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 00:26:48.544105 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8322,"thread_start_us":7763,"threads_started":1}
I20260811 00:26:48.544766 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:48.525397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:32855 (local address 127.0.0.1:36700)
0811 00:26:48.530875 (+ 5478us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:48.530913 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:48.531136 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:48.535897 (+ 4761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:48.535905 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:48.535967 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:48.537204 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:48.537228 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:48.539252 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:48.539257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:48.541678 (+ 2421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:48.541698 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:48.543547 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:48.543554 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:48.543759 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:48.543799 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 00:26:48.543903 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 00:26:48.544090 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5288,"mutex_wait_us":4,"spinlock_wait_cycles":2432,"thread_start_us":8591,"threads_started":1}
I20260811 00:26:48.544708 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:48.525395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:46301 (local address 127.0.0.1:49058)
0811 00:26:48.532585 (+ 7190us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:48.532618 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:48.532904 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:48.535006 (+ 2102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:48.535016 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:48.535067 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:48.535655 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:48.535677 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:48.539452 (+ 3775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:48.539458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:48.541580 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:48.541603 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:48.543546 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:48.543553 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:48.543769 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:48.543794 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 00:26:48.543864 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 00:26:48.544058 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6954,"mutex_wait_us":5,"thread_start_us":6214,"threads_started":1}
I20260811 00:26:48.545697 10442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f58553db3fb46c49fa58a3ea3838cba" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e60785b019bf4273960830aeeb93fd30" is_pre_election: true
I20260811 00:26:48.546057 11035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:48.532610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49058 (local address 127.2.120.61:46301)
0811 00:26:48.534565 (+ 1955us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:48.534575 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:48.534603 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:26:48.534678 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:48.534683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:48.534767 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:48.535909 (+ 1142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:48.537737 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:48.539251 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:48.541850 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:48.543301 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:48.545190 (+ 1889us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:48.545323 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:48.545373 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 00:26:48.545552 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":698,"server-negotiator.queue_time_us":877,"spinlock_wait_cycles":26240,"thread_start_us":537,"threads_started":1}
I20260811 00:26:48.546195 10442 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f58553db3fb46c49fa58a3ea3838cba in term 1.
I20260811 00:26:48.546927 10205 leader_election.cc:302] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f58553db3fb46c49fa58a3ea3838cba, e60785b019bf4273960830aeeb93fd30; no voters: 97e4014ce0a74657ab9899f0ba4c2eee
I20260811 00:26:48.546977 10233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e60785b019bf4273960830aeeb93fd30" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5f58553db3fb46c49fa58a3ea3838cba" is_pre_election: true
I20260811 00:26:48.547325 10233 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e60785b019bf4273960830aeeb93fd30 in term 1.
I20260811 00:26:48.547528 11028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 00:26:48.547771 11028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 1 FOLLOWER]: Starting leader election (detected failure of leader 97e4014ce0a74657ab9899f0ba4c2eee)
I20260811 00:26:48.547914 11028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 1 FOLLOWER]: Advancing to term 2
I20260811 00:26:48.548182 10415 leader_election.cc:302] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f58553db3fb46c49fa58a3ea3838cba, e60785b019bf4273960830aeeb93fd30; no voters: 97e4014ce0a74657ab9899f0ba4c2eee
I20260811 00:26:48.548844 11026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 00:26:48.548992 11026 raft_consensus.cc:482] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 97e4014ce0a74657ab9899f0ba4c2eee)
I20260811 00:26:48.549173 11026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 1 FOLLOWER]: Advancing to term 2
I20260811 00:26:48.550459 11028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } }
I20260811 00:26:48.551167 11028 leader_election.cc:288] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [CANDIDATE]: Term 2 election: Requested vote from peers 97e4014ce0a74657ab9899f0ba4c2eee (127.2.120.62:33143), e60785b019bf4273960830aeeb93fd30 (127.2.120.60:32855)
I20260811 00:26:48.552230 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:48.551599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:44256)
0811 00:26:48.551885 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:48.552005 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 00:26:48.552443 10442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f58553db3fb46c49fa58a3ea3838cba" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e60785b019bf4273960830aeeb93fd30"
I20260811 00:26:48.552631 11026 raft_consensus.cc:504] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } }
I20260811 00:26:48.553206 10442 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5f58553db3fb46c49fa58a3ea3838cba in current term 2: Already voted for candidate e60785b019bf4273960830aeeb93fd30 in this term.
I20260811 00:26:48.553509 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:48.553019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:44268)
0811 00:26:48.553210 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:48.553337 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 00:26:48.553529 11026 leader_election.cc:288] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [CANDIDATE]: Term 2 election: Requested vote from peers 97e4014ce0a74657ab9899f0ba4c2eee (127.2.120.62:33143), 5f58553db3fb46c49fa58a3ea3838cba (127.2.120.61:46301)
W20260811 00:26:48.554194 10205 leader_election.cc:334] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 97e4014ce0a74657ab9899f0ba4c2eee (127.2.120.62:33143): Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
I20260811 00:26:48.554263 10233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e60785b019bf4273960830aeeb93fd30" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5f58553db3fb46c49fa58a3ea3838cba"
I20260811 00:26:48.554570 10233 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e60785b019bf4273960830aeeb93fd30 in current term 2: Already voted for candidate 5f58553db3fb46c49fa58a3ea3838cba in this term.
I20260811 00:26:48.554596 10205 leader_election.cc:302] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f58553db3fb46c49fa58a3ea3838cba; no voters: 97e4014ce0a74657ab9899f0ba4c2eee, e60785b019bf4273960830aeeb93fd30
I20260811 00:26:48.554575 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:48.553931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:44284)
0811 00:26:48.554118 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:48.554209 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 00:26:48.555078 11028 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 00:26:48.555981 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:48.555557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:44294)
0811 00:26:48.555720 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:48.555816 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 00:26:48.556210 10414 leader_election.cc:334] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 97e4014ce0a74657ab9899f0ba4c2eee (127.2.120.62:33143): Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
I20260811 00:26:48.556368 10414 leader_election.cc:302] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e60785b019bf4273960830aeeb93fd30; no voters: 5f58553db3fb46c49fa58a3ea3838cba, 97e4014ce0a74657ab9899f0ba4c2eee
I20260811 00:26:48.556764 11026 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 00:26:49.001812 11039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:49.001028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.2:59315)
0811 00:26:49.001517 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:49.001610 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":251,"threads_started":1}
I20260811 00:26:49.004027 11041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:49.002976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.3:41417)
0811 00:26:49.003509 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:49.003699 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":312,"threads_started":1}
I20260811 00:26:49.010483 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:49.009607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.1:38851)
0811 00:26:49.010091 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:49.010235 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":255,"threads_started":1}
I20260811 00:26:50.004213 11043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.003193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.2:47501)
0811 00:26:50.003842 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.003967 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":377,"threads_started":1}
I20260811 00:26:50.005858 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.004980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.3:44723)
0811 00:26:50.005510 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.005638 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":308,"threads_started":1}
I20260811 00:26:50.012631 11045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.011655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.1:52313)
0811 00:26:50.012271 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.012368 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":352,"threads_started":1}
I20260811 00:26:50.168876 11046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:26:50.169191 11046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } }
I20260811 00:26:50.169938 11046 leader_election.cc:288] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 97e4014ce0a74657ab9899f0ba4c2eee (127.2.120.62:33143), e60785b019bf4273960830aeeb93fd30 (127.2.120.60:32855)
I20260811 00:26:50.171332 11047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.170382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:44566)
0811 00:26:50.170995 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.171134 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":316,"threads_started":1}
I20260811 00:26:50.171916 10442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f58553db3fb46c49fa58a3ea3838cba" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e60785b019bf4273960830aeeb93fd30" is_pre_election: true
I20260811 00:26:50.172273 10442 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f58553db3fb46c49fa58a3ea3838cba in term 2.
I20260811 00:26:50.173023 10205 leader_election.cc:302] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f58553db3fb46c49fa58a3ea3838cba, e60785b019bf4273960830aeeb93fd30; no voters:
I20260811 00:26:50.173266 11047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.172749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:44574)
0811 00:26:50.172923 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.173014 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 00:26:50.173382 11046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260811 00:26:50.173492 10205 leader_election.cc:334] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 97e4014ce0a74657ab9899f0ba4c2eee (127.2.120.62:33143): Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
I20260811 00:26:50.173574 11046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:26:50.173704 11046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 2 FOLLOWER]: Advancing to term 3
I20260811 00:26:50.176046 11046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } }
I20260811 00:26:50.176832 11046 leader_election.cc:288] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [CANDIDATE]: Term 3 election: Requested vote from peers 97e4014ce0a74657ab9899f0ba4c2eee (127.2.120.62:33143), e60785b019bf4273960830aeeb93fd30 (127.2.120.60:32855)
I20260811 00:26:50.177497 11047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.177081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:44586)
0811 00:26:50.177262 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.177344 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 00:26:50.177779 10442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f58553db3fb46c49fa58a3ea3838cba" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e60785b019bf4273960830aeeb93fd30"
I20260811 00:26:50.178009 10442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 2 FOLLOWER]: Advancing to term 3
I20260811 00:26:50.179488 11047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.178826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:44588)
0811 00:26:50.179137 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.179297 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260811 00:26:50.179848 10205 leader_election.cc:334] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 97e4014ce0a74657ab9899f0ba4c2eee (127.2.120.62:33143): Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
I20260811 00:26:50.181368 10442 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f58553db3fb46c49fa58a3ea3838cba in term 3.
I20260811 00:26:50.182070 10205 leader_election.cc:302] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f58553db3fb46c49fa58a3ea3838cba, e60785b019bf4273960830aeeb93fd30; no voters: 97e4014ce0a74657ab9899f0ba4c2eee
I20260811 00:26:50.182794 11046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 3 FOLLOWER]: Leader election won for term 3
I20260811 00:26:50.183316 11046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [term 3 LEADER]: Becoming Leader. State: Replica: 5f58553db3fb46c49fa58a3ea3838cba, State: Running, Role: LEADER
I20260811 00:26:50.183810 11046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } }
I20260811 00:26:50.187139 11049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f58553db3fb46c49fa58a3ea3838cba. Latest consensus state: current_term: 3 leader_uuid: "5f58553db3fb46c49fa58a3ea3838cba" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } } }
I20260811 00:26:50.187490 11049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:50.188360 11051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:26:50.193181 11051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:26:50.194392 11051 catalog_manager.cc:1259] Loaded cluster ID: be18ea85b5ef4dc29a31e146ebce0964
I20260811 00:26:50.194517 11051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:26:50.195887 11051 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:26:50.196918 11051 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: Loaded TSK: 0
I20260811 00:26:50.197950 11051 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:26:50.206221 10221 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:26:50.213763 10221 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:26:50.219596 11047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.218923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:44614)
0811 00:26:50.219141 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.219269 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 00:26:50.221127 10442 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [term 3 FOLLOWER]: Refusing update from remote peer 5f58553db3fb46c49fa58a3ea3838cba: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 00:26:50.222074 11047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.221363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:44630)
0811 00:26:50.221753 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.221880 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207}
I20260811 00:26:50.222470 11049 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [LEADER]: Connected to new peer: Peer: permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260811 00:26:50.222965 10205 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba -> Peer 97e4014ce0a74657ab9899f0ba4c2eee (127.2.120.62:33143): Couldn't send request to peer 97e4014ce0a74657ab9899f0ba4c2eee. Status: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 00:26:50.226588 11053 sys_catalog.cc:455] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f58553db3fb46c49fa58a3ea3838cba. Latest consensus state: current_term: 3 leader_uuid: "5f58553db3fb46c49fa58a3ea3838cba" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } } }
I20260811 00:26:50.227020 11053 sys_catalog.cc:458] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:26:50.229090 11046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "5f58553db3fb46c49fa58a3ea3838cba" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } } }
I20260811 00:26:50.229511 11046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:50.229801 11053 sys_catalog.cc:455] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "5f58553db3fb46c49fa58a3ea3838cba" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "97e4014ce0a74657ab9899f0ba4c2eee" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33143 } } peers { permanent_uuid: "5f58553db3fb46c49fa58a3ea3838cba" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 46301 } attrs { promote: false } } peers { permanent_uuid: "e60785b019bf4273960830aeeb93fd30" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 32855 } attrs { promote: false } } }
I20260811 00:26:50.230131 11053 sys_catalog.cc:458] T 00000000000000000000000000000000 P e60785b019bf4273960830aeeb93fd30 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:26:50.230650 11056 mvcc.cc:205] Tried to move back new op lower bound from 7317127209849823232 to 7317127209716875264. Current Snapshot: MvccSnapshot[applied={T|T < 7317127209849823232 or (T in {7317127209849823232})}]
I20260811 00:26:50.253851 11047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.240503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:38089 (local address 127.0.0.1:41864)
0811 00:26:50.241341 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.241380 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:50.241532 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:50.242686 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:50.242691 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:50.242729 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:50.243104 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:50.243123 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.244502 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.244539 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:50.247541 (+ 3002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:50.247568 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.250528 (+ 2960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.250539 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:50.251870 (+ 1331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:50.251953 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 00:26:50.252142 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 00:26:50.252422 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":253}
I20260811 00:26:50.257889 11063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.241687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42268 (local address 127.2.120.3:43095)
0811 00:26:50.242293 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:50.242301 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:50.245456 (+ 3155us) server_negotiation.cc:407] Connection header received
0811 00:26:50.245927 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:50.245938 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:50.246068 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:50.246184 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:50.247639 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.249485 (+ 1846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.254464 (+ 4979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.256352 (+ 1888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.256465 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:50.256657 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:50.257332 (+ 675us) server_negotiation.cc:299] Negotiation successful
0811 00:26:50.257504 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":507,"threads_started":1}
I20260811 00:26:50.257932 11062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.241018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:43095 (local address 127.0.0.1:42268)
0811 00:26:50.245337 (+ 4319us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.245360 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:50.245585 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:50.246589 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:50.246597 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:50.246653 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:50.247294 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:50.247315 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.249876 (+ 2561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.249887 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:50.254078 (+ 4191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:50.254116 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.256805 (+ 2689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.256811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:50.257100 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:50.257148 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 00:26:50.257276 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 00:26:50.257448 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4124,"mutex_wait_us":39,"thread_start_us":3922,"threads_started":1}
I20260811 00:26:50.258850 11061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.240826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41864 (local address 127.2.120.1:38089)
0811 00:26:50.242330 (+ 1504us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:50.242337 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:50.242359 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:26:50.242441 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:50.242445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:50.242509 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:50.242609 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:50.243318 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.244342 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.247914 (+ 3572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.250133 (+ 2219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.250971 (+ 838us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:50.251178 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:50.258221 (+ 7043us) server_negotiation.cc:299] Negotiation successful
0811 00:26:50.258431 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1370,"thread_start_us":1201,"threads_started":1}
I20260811 00:26:50.259335 10869 tablet_service.cc:1467] Processing CreateTablet for tablet d7605d75a65746009b2c6e363413a665 (DEFAULT_TABLE table=test-table [id=8f9c608e1879444a976591b8b30ac545]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:50.259833 10559 tablet_service.cc:1467] Processing CreateTablet for tablet d7605d75a65746009b2c6e363413a665 (DEFAULT_TABLE table=test-table [id=8f9c608e1879444a976591b8b30ac545]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:50.260005 10869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7605d75a65746009b2c6e363413a665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:50.260396 10559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7605d75a65746009b2c6e363413a665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:50.263428 11047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.245464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:45043 (local address 127.0.0.1:48960)
0811 00:26:50.254994 (+ 9530us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.255023 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:50.255213 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:50.256723 (+ 1510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:50.256728 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:50.256767 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:50.256994 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:50.257011 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.258514 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.258521 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:50.261036 (+ 2515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:50.261065 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.262470 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.262478 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:50.262703 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:50.262734 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 00:26:50.262862 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 00:26:50.263033 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9326,"thread_start_us":10113,"threads_started":1}
I20260811 00:26:50.264060 11065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.245635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48960 (local address 127.2.120.2:45043)
0811 00:26:50.256256 (+ 10621us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:50.256263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:50.256289 (+ 26us) server_negotiation.cc:407] Connection header received
0811 00:26:50.256375 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:50.256381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:50.256472 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:50.257179 (+ 707us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:50.257372 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.258317 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.261257 (+ 2940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.262252 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.262442 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:50.262586 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:50.262909 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 00:26:50.263095 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10461,"thread_start_us":10321,"threads_started":1}
I20260811 00:26:50.265573 10717 tablet_service.cc:1467] Processing CreateTablet for tablet d7605d75a65746009b2c6e363413a665 (DEFAULT_TABLE table=test-table [id=8f9c608e1879444a976591b8b30ac545]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:50.266108 10717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7605d75a65746009b2c6e363413a665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:50.273420 11067 tablet_bootstrap.cc:492] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace: Bootstrap starting.
I20260811 00:26:50.274477 11066 tablet_bootstrap.cc:492] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb: Bootstrap starting.
I20260811 00:26:50.276990 11066 tablet_bootstrap.cc:654] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:50.277215 11068 tablet_bootstrap.cc:492] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d: Bootstrap starting.
I20260811 00:26:50.277456 11067 tablet_bootstrap.cc:654] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:50.279915 11068 tablet_bootstrap.cc:654] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:50.283370 11066 tablet_bootstrap.cc:492] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb: No bootstrap required, opened a new log
I20260811 00:26:50.283582 11066 ts_tablet_manager.cc:1397] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 00:26:50.284385 11066 raft_consensus.cc:348] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } }
I20260811 00:26:50.284719 11066 raft_consensus.cc:374] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:50.284884 11066 raft_consensus.cc:729] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0691687a81d4375ac1258533cdce5eb, State: Initialized, Role: FOLLOWER
I20260811 00:26:50.285262 11066 consensus_queue.cc:260] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } }
I20260811 00:26:50.285799 11067 tablet_bootstrap.cc:492] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace: No bootstrap required, opened a new log
I20260811 00:26:50.286001 11067 ts_tablet_manager.cc:1397] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20260811 00:26:50.286405 11066 ts_tablet_manager.cc:1428] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 00:26:50.287002 11067 raft_consensus.cc:348] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } }
I20260811 00:26:50.287290 11067 raft_consensus.cc:374] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:50.287338 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.286833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.3:47195)
0811 00:26:50.287037 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.287155 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 00:26:50.287406 11067 raft_consensus.cc:729] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dcaa36e9c34456da1814e7cc9e27ace, State: Initialized, Role: FOLLOWER
I20260811 00:26:50.287673 11067 consensus_queue.cc:260] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } }
I20260811 00:26:50.289916 11068 tablet_bootstrap.cc:492] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d: No bootstrap required, opened a new log
I20260811 00:26:50.290128 11068 ts_tablet_manager.cc:1397] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.007s
I20260811 00:26:50.290232 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.288117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.3:46497)
0811 00:26:50.289876 (+ 1759us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.290003 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1591}
I20260811 00:26:50.290287 10943 heartbeater.cc:503] Master 127.2.120.61:46301 was elected leader, sending a full tablet report...
I20260811 00:26:50.291208 11068 raft_consensus.cc:348] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } }
I20260811 00:26:50.291445 11068 raft_consensus.cc:374] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:50.291512 11068 raft_consensus.cc:729] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2150e97860df4cc7bbfbf9b18dc8f86d, State: Initialized, Role: FOLLOWER
I20260811 00:26:50.291863 11068 consensus_queue.cc:260] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } }
I20260811 00:26:50.292902 11067 ts_tablet_manager.cc:1428] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace: Time spent starting tablet: real 0.007s user 0.000s sys 0.004s
I20260811 00:26:50.293344 11045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.292073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.1:57963)
0811 00:26:50.293034 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.293172 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":819}
I20260811 00:26:50.293502 11068 ts_tablet_manager.cc:1428] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 00:26:50.295022 11045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.294157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.1:57429)
0811 00:26:50.294729 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.294831 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447}
I20260811 00:26:50.295985 11043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.295434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.2:40705)
0811 00:26:50.295655 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.295779 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 00:26:50.296854 10630 heartbeater.cc:503] Master 127.2.120.61:46301 was elected leader, sending a full tablet report...
I20260811 00:26:50.564235 11074 raft_consensus.cc:482] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:26:50.564590 11074 raft_consensus.cc:504] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } }
I20260811 00:26:50.566401 11074 leader_election.cc:288] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dcaa36e9c34456da1814e7cc9e27ace (127.2.120.1:38089), e0691687a81d4375ac1258533cdce5eb (127.2.120.3:43095)
I20260811 00:26:50.574270 11061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.567144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:54761 (local address 127.2.120.1:38089)
0811 00:26:50.567398 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:50.567406 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:50.567742 (+ 336us) server_negotiation.cc:407] Connection header received
0811 00:26:50.567875 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:50.567885 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:50.568013 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:50.568213 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:50.569352 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.570242 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.572199 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.573136 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.573325 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:50.573451 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:50.573828 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 00:26:50.573975 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 00:26:50.574291 11043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.567107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:38089 (local address 127.2.120.2:54761)
0811 00:26:50.567371 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.567401 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:50.567635 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:50.568285 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:50.568292 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:50.568339 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:50.569083 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:50.569111 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.570451 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.570458 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:50.571973 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:50.571990 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.573444 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.573451 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:50.573637 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:50.573666 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 00:26:50.573794 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 00:26:50.573959 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 00:26:50.575412 11063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.567131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:44521 (local address 127.2.120.3:43095)
0811 00:26:50.567648 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:50.567658 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:50.569217 (+ 1559us) server_negotiation.cc:407] Connection header received
0811 00:26:50.569626 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:50.569651 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:50.569802 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:50.569961 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:50.570657 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.571542 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.573513 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.574391 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.574566 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:50.574674 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:50.575099 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 00:26:50.575218 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328}
I20260811 00:26:50.575636 11075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.567120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:43095 (local address 127.2.120.2:44521)
0811 00:26:50.569067 (+ 1947us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.569088 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:50.569292 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:50.569993 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:50.569999 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:50.570030 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:50.570446 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:50.570458 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.571808 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.571813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:50.573290 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:50.573306 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.574756 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.574763 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:50.574903 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:50.574934 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 00:26:50.575014 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 00:26:50.575189 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1777,"mutex_wait_us":119,"thread_start_us":1414,"threads_started":1}
I20260811 00:26:50.575497 10579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7605d75a65746009b2c6e363413a665" candidate_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" is_pre_election: true
I20260811 00:26:50.576066 10579 raft_consensus.cc:2454] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2150e97860df4cc7bbfbf9b18dc8f86d in term 0.
I20260811 00:26:50.576637 10891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7605d75a65746009b2c6e363413a665" candidate_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0691687a81d4375ac1258533cdce5eb" is_pre_election: true
I20260811 00:26:50.576970 10891 raft_consensus.cc:2454] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2150e97860df4cc7bbfbf9b18dc8f86d in term 0.
I20260811 00:26:50.576925 10666 leader_election.cc:302] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dcaa36e9c34456da1814e7cc9e27ace, 2150e97860df4cc7bbfbf9b18dc8f86d; no voters:
I20260811 00:26:50.577666 11074 raft_consensus.cc:2781] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:26:50.578083 11074 raft_consensus.cc:482] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:26:50.578274 11074 raft_consensus.cc:3037] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:50.580804 11074 raft_consensus.cc:504] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } }
I20260811 00:26:50.581583 11074 leader_election.cc:288] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [CANDIDATE]: Term 1 election: Requested vote from peers 0dcaa36e9c34456da1814e7cc9e27ace (127.2.120.1:38089), e0691687a81d4375ac1258533cdce5eb (127.2.120.3:43095)
I20260811 00:26:50.582330 10891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7605d75a65746009b2c6e363413a665" candidate_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0691687a81d4375ac1258533cdce5eb"
I20260811 00:26:50.582366 10579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7605d75a65746009b2c6e363413a665" candidate_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dcaa36e9c34456da1814e7cc9e27ace"
I20260811 00:26:50.582610 10891 raft_consensus.cc:3037] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:50.582655 10579 raft_consensus.cc:3037] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:50.585150 10579 raft_consensus.cc:2454] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2150e97860df4cc7bbfbf9b18dc8f86d in term 1.
I20260811 00:26:50.585922 10666 leader_election.cc:302] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dcaa36e9c34456da1814e7cc9e27ace, 2150e97860df4cc7bbfbf9b18dc8f86d; no voters:
I20260811 00:26:50.586323 10891 raft_consensus.cc:2454] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2150e97860df4cc7bbfbf9b18dc8f86d in term 1.
I20260811 00:26:50.586410 11074 raft_consensus.cc:2781] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:50.587132 11074 raft_consensus.cc:686] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [term 1 LEADER]: Becoming Leader. State: Replica: 2150e97860df4cc7bbfbf9b18dc8f86d, State: Running, Role: LEADER
I20260811 00:26:50.587702 11074 consensus_queue.cc:237] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } }
I20260811 00:26:50.588095 10787 heartbeater.cc:503] Master 127.2.120.61:46301 was elected leader, sending a full tablet report...
I20260811 00:26:50.589395 11075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.588426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.2:49269)
0811 00:26:50.588969 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.589125 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327}
I20260811 00:26:50.592885 10221 catalog_manager.cc:5495] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2150e97860df4cc7bbfbf9b18dc8f86d (127.2.120.2). New cstate: current_term: 1 leader_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2150e97860df4cc7bbfbf9b18dc8f86d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45043 } health_report { overall_health: HEALTHY } } }
I20260811 00:26:50.668717 11065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.652263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48964 (local address 127.2.120.2:45043)
0811 00:26:50.653705 (+ 1442us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:50.653712 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:50.653746 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:26:50.653970 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:50.653979 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:50.654268 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:50.654684 (+ 416us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:26:50.655835 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.657580 (+ 1745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.661653 (+ 4073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.663141 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.663612 (+ 471us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:50.665146 (+ 1534us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:26:50.667078 (+ 1932us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:26:50.667616 (+ 538us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:50.667923 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 00:26:50.668104 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1049}
I20260811 00:26:50.679350 10891 raft_consensus.cc:1261] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb [term 1 FOLLOWER]: Refusing update from remote peer 2150e97860df4cc7bbfbf9b18dc8f86d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:50.680323 10579 raft_consensus.cc:1261] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace [term 1 FOLLOWER]: Refusing update from remote peer 2150e97860df4cc7bbfbf9b18dc8f86d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:50.680801 11076 consensus_queue.cc:1035] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0691687a81d4375ac1258533cdce5eb" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:50.682049 11074 consensus_queue.cc:1035] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dcaa36e9c34456da1814e7cc9e27ace" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:50.686479 11045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.685541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.1:52235)
0811 00:26:50.685989 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.686152 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249}
I20260811 00:26:50.688764 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.687433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.3:46081)
0811 00:26:50.688045 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.688268 (+ 223us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341}
I20260811 00:26:50.699080 11045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.698540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.1:59639)
0811 00:26:50.698728 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.698840 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 00:26:50.709904 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.709138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.3:35239)
0811 00:26:50.709574 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.709683 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296}
I20260811 00:26:50.712581 11075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.711349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.2:49183)
0811 00:26:50.712155 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.712296 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":633}
I20260811 00:26:50.717856 11075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.717213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.2.120.2:45875)
0811 00:26:50.717492 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.717635 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260811 00:26:50.734346 10220 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:50.734676 10220 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:50.748216 10220 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f58553db3fb46c49fa58a3ea3838cba: Sending DeleteTablet for 3 replicas of tablet d7605d75a65746009b2c6e363413a665
I20260811 00:26:50.749810 10869 tablet_service.cc:1514] Processing DeleteTablet for tablet d7605d75a65746009b2c6e363413a665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:50 UTC) from {username='oryx', principal='oryx/127.2.120.61@KRBTEST.COM'} at 127.0.0.1:42268
I20260811 00:26:50.749871 10868 tablet_service.cc:1514] Processing DeleteTablet for tablet d7605d75a65746009b2c6e363413a665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:50 UTC) from {username='oryx', principal='oryx/127.2.120.61@KRBTEST.COM'} at 127.0.0.1:42268
I20260811 00:26:50.750169 10717 tablet_service.cc:1514] Processing DeleteTablet for tablet d7605d75a65746009b2c6e363413a665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:50 UTC) from {username='oryx', principal='oryx/127.2.120.61@KRBTEST.COM'} at 127.0.0.1:48960
I20260811 00:26:50.751014 11092 tablet_replica.cc:330] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d: stopping tablet replica
I20260811 00:26:50.751305 11092 raft_consensus.cc:2229] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:26:50.751597 10559 tablet_service.cc:1514] Processing DeleteTablet for tablet d7605d75a65746009b2c6e363413a665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:50 UTC) from {username='oryx', principal='oryx/127.2.120.61@KRBTEST.COM'} at 127.0.0.1:41864
I20260811 00:26:50.751964 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 10402
I20260811 00:26:50.752336 11092 raft_consensus.cc:2258] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:50.756409 11092 ts_tablet_manager.cc:1905] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:50.760766 11093 tablet_replica.cc:330] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace: stopping tablet replica
I20260811 00:26:50.761087 11091 tablet_replica.cc:330] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb: stopping tablet replica
I20260811 00:26:50.761050 11093 raft_consensus.cc:2229] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:50.761302 11091 raft_consensus.cc:2229] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:50.761466 11093 raft_consensus.cc:2258] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:50.761637 11091 raft_consensus.cc:2258] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:50.763317 11093 ts_tablet_manager.cc:1905] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:50.765177 11091 ts_tablet_manager.cc:1905] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:50.772859 11093 ts_tablet_manager.cc:1918] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:26:50.773039 11093 log.cc:1196] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/d7605d75a65746009b2c6e363413a665
I20260811 00:26:50.773568 11093 ts_tablet_manager.cc:1939] T d7605d75a65746009b2c6e363413a665 P 0dcaa36e9c34456da1814e7cc9e27ace: Deleting consensus metadata
I20260811 00:26:50.773702 11091 ts_tablet_manager.cc:1918] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:26:50.773864 11091 log.cc:1196] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/d7605d75a65746009b2c6e363413a665
I20260811 00:26:50.774410 11091 ts_tablet_manager.cc:1939] T d7605d75a65746009b2c6e363413a665 P e0691687a81d4375ac1258533cdce5eb: Deleting consensus metadata
I20260811 00:26:50.775171 10207 catalog_manager.cc:4841] TS 0dcaa36e9c34456da1814e7cc9e27ace (127.2.120.1:38089): tablet d7605d75a65746009b2c6e363413a665 (table test-table [id=8f9c608e1879444a976591b8b30ac545]) successfully deleted
I20260811 00:26:50.775771 10208 catalog_manager.cc:4841] TS e0691687a81d4375ac1258533cdce5eb (127.2.120.3:43095): tablet d7605d75a65746009b2c6e363413a665 (table test-table [id=8f9c608e1879444a976591b8b30ac545]) successfully deleted
W20260811 00:26:50.776463 10208 catalog_manager.cc:4808] TS e0691687a81d4375ac1258533cdce5eb (127.2.120.3:43095): delete failed for tablet d7605d75a65746009b2c6e363413a665 because the tablet was not found. No further retry: Not found: Tablet not found: d7605d75a65746009b2c6e363413a665
I20260811 00:26:50.777956 11061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.754040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41880 (local address 127.2.120.1:38089)
0811 00:26:50.754318 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:50.754328 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:50.754509 (+ 181us) server_negotiation.cc:407] Connection header received
0811 00:26:50.754866 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:50.754872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:50.754962 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:50.755063 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:50.757473 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.758683 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.760696 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.761653 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.761791 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:50.767042 (+ 5251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:50.767086 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:50.767121 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:50.767191 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:50.772579 (+ 5388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:50.774373 (+ 1794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:50.774384 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:50.774388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:50.774686 (+ 298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:50.775246 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:50.775265 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:50.775269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:50.775796 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:50.776152 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:50.776904 (+ 752us) server_negotiation.cc:299] Negotiation successful
0811 00:26:50.777295 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 00:26:50.779673 10621 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:50.782956 11092 ts_tablet_manager.cc:1918] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:26:50.783186 11092 log.cc:1196] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/d7605d75a65746009b2c6e363413a665
I20260811 00:26:50.783862 11092 ts_tablet_manager.cc:1939] T d7605d75a65746009b2c6e363413a665 P 2150e97860df4cc7bbfbf9b18dc8f86d: Deleting consensus metadata
I20260811 00:26:50.785535 10208 catalog_manager.cc:4841] TS 2150e97860df4cc7bbfbf9b18dc8f86d (127.2.120.2:45043): tablet d7605d75a65746009b2c6e363413a665 (table test-table [id=8f9c608e1879444a976591b8b30ac545]) successfully deleted
I20260811 00:26:50.789865 11095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.764355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:44632)
0811 00:26:50.789517 (+ 25162us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.789653 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25010,"threads_started":1}
I20260811 00:26:50.791769 11095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.791376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:44644)
0811 00:26:50.791550 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:50.791637 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 00:26:50.953961 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 10402
I20260811 00:26:50.984658 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 10640
I20260811 00:26:51.001919 11065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:50.985595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48978 (local address 127.2.120.2:45043)
0811 00:26:50.985833 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:50.985840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:50.985930 (+ 90us) server_negotiation.cc:407] Connection header received
0811 00:26:50.986110 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:50.986117 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:50.986244 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:50.986389 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:50.988114 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.989417 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.990923 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:50.991733 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:50.991936 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:50.994936 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:50.994960 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:50.994980 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:50.995045 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:50.998565 (+ 3520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:50.999414 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:50.999424 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:50.999426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:50.999618 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:51.000044 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:51.000051 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:51.000057 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:51.000480 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:51.000923 (+ 443us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:51.001223 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 00:26:51.001480 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 00:26:51.002779 10780 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:51.129716 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 10640
I20260811 00:26:51.155170 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 10798
I20260811 00:26:51.172260 11099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:51.156444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42284 (local address 127.2.120.3:43095)
0811 00:26:51.157363 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:51.157368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:51.157391 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:26:51.157461 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:51.157465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:51.157569 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:51.157673 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:51.159139 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:51.159900 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:51.161569 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:51.162085 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:51.162222 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:51.165398 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:51.165421 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:51.165440 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:51.165486 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:51.168662 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:51.169564 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:51.169573 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:51.169575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:51.169742 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:51.170138 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:51.170142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:51.170161 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:51.170607 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:51.170880 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:51.171218 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 00:26:51.171453 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":400,"threads_started":1}
I20260811 00:26:51.173522 10935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:51.228330 11095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:51.227841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:44646)
0811 00:26:51.228046 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:51.228139 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 00:26:51.229840 11095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:51.229167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33143 (local address 127.0.0.1:44650)
0811 00:26:51.229489 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:51.229620 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:33143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260811 00:26:51.279223 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 10798
I20260811 00:26:51.303704 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 10069
I20260811 00:26:51.305402 10254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:51.439641 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 10069
W20260811 00:26:51.469218 10417 connection.cc:535] server connection from 127.0.0.1:36700 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 00:26:51.469710 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 10177
I20260811 00:26:51.471135 10463 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:51.596983 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 10177
2026-08-11T00:26:51Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (12762 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:26:51 dist-test-slave-fd7l krb5kdc[11105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:26:51 dist-test-slave-fd7l krb5kdc[11105](info): set up 2 sockets
Aug 11 00:26:51 dist-test-slave-fd7l krb5kdc[11105](info): commencing operation
krb5kdc: starting...
W20260811 00:26:52.834997 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.005s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:26:53 dist-test-slave-fd7l krb5kdc[11105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:26:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:26:53Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.62.keytab.
Entry for principal oryx/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.62.keytab.
I20260811 00:26:53.143586 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:39689
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.62.keytab
--principal=oryx/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:39689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:53.552416 11120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:53.552798 11120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:53.553001 11120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:53.562317 11120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:26:53.562477 11120 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 00:26:53.562570 11120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:53.562700 11120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:53.562789 11120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:26:53.562902 11120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:26:53.574287 11120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:39689
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.62.keytab
--principal=oryx/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:39689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:53.575862 11120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:53.577557 11120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:53.593497 11128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:53.593495 11126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:53.594034 11125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:53.594789 11120 server_base.cc:1034] running on GCE node
I20260811 00:26:53.595928 11120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:53.598476 11120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:53.599789 11120 hybrid_clock.cc:648] HybridClock initialized: now 1786408013599698 us; error 73 us; skew 500 ppm
I20260811 00:26:53.600286 11120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:53 dist-test-slave-fd7l krb5kdc[11105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408013, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:53.604331 11120 init.cc:377] Logged in from keytab as oryx/127.2.120.62@KRBTEST.COM (short username oryx)
I20260811 00:26:53.606380 11120 webserver.cc:460] Webserver started at http://127.2.120.62:39921/ using document root <none> and password file <none>
I20260811 00:26:53.607245 11120 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:53.607357 11120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:53.607708 11120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:53.611284 11120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "81df64d70ee74d6dbfb37064593c0b9a"
format_stamp: "Formatted at 2026-08-11 00:26:53 on dist-test-slave-fd7l"
server_key: "abe802600280fc95df075784ced7b286"
server_key_iv: "9ed27b81c3197e50936380486a25d383"
server_key_version: "encryptionkey@0"
I20260811 00:26:53.612376 11120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "81df64d70ee74d6dbfb37064593c0b9a"
format_stamp: "Formatted at 2026-08-11 00:26:53 on dist-test-slave-fd7l"
server_key: "abe802600280fc95df075784ced7b286"
server_key_iv: "9ed27b81c3197e50936380486a25d383"
server_key_version: "encryptionkey@0"
I20260811 00:26:53.621821 11120 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.000s
I20260811 00:26:53.630584 11135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:53.635133 11120 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.005s sys 0.000s
I20260811 00:26:53.635758 11120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "81df64d70ee74d6dbfb37064593c0b9a"
format_stamp: "Formatted at 2026-08-11 00:26:53 on dist-test-slave-fd7l"
server_key: "abe802600280fc95df075784ced7b286"
server_key_iv: "9ed27b81c3197e50936380486a25d383"
server_key_version: "encryptionkey@0"
I20260811 00:26:53.636363 11120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:53.658130 11120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:53.664652 11120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:53.664978 11120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:53.692315 11120 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:39689
I20260811 00:26:53.692342 11203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:39689 every 8 connection(s)
I20260811 00:26:53.694765 11120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:26:53.702023 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 11120
I20260811 00:26:53.702421 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:26:53.702838 11205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:53.704027 2528 external_mini_cluster.cc:1428] Setting key 81c2284a28aad6bff52d7daee4fd98ac
I20260811 00:26:53.732499 11205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a: Bootstrap starting.
Aug 11 00:26:53 dist-test-slave-fd7l krb5kdc[11105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM
I20260811 00:26:53.740162 11205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:53.742852 11205 log.cc:824] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:53.752924 11205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a: No bootstrap required, opened a new log
I20260811 00:26:53.757302 11208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:53.708179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57252 (local address 127.2.120.62:39689)
0811 00:26:53.709794 (+ 1615us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:53.709828 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:53.709915 (+ 87us) server_negotiation.cc:407] Connection header received
0811 00:26:53.713190 (+ 3275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:53.713243 (+ 53us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:53.714110 (+ 867us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:53.714837 (+ 727us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:53.718642 (+ 3805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:53.721108 (+ 2466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:53.724467 (+ 3359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:53.726301 (+ 1834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:53.726622 (+ 321us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:53.735295 (+ 8673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:53.735324 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:53.735375 (+ 51us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:53.735499 (+ 124us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:53.742953 (+ 7454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:53.745787 (+ 2834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:53.745814 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:53.745830 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:53.746514 (+ 684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:53.747730 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:53.747738 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:53.747742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:53.750030 (+ 2288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:53.751194 (+ 1164us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:53.752181 (+ 987us) server_negotiation.cc:299] Negotiation successful
0811 00:26:53.753332 (+ 1151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1176,"thread_start_us":795,"threads_started":1}
I20260811 00:26:53.775652 11205 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81df64d70ee74d6dbfb37064593c0b9a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39689 } }
I20260811 00:26:53.776392 11205 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:53.776492 11205 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81df64d70ee74d6dbfb37064593c0b9a, State: Initialized, Role: FOLLOWER
I20260811 00:26:53.777725 11205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81df64d70ee74d6dbfb37064593c0b9a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39689 } }
I20260811 00:26:53.778249 11205 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:26:53.778560 11205 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:26:53.778937 11205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:53.783166 11205 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81df64d70ee74d6dbfb37064593c0b9a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39689 } }
I20260811 00:26:53.784278 11205 leader_election.cc:302] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81df64d70ee74d6dbfb37064593c0b9a; no voters:
I20260811 00:26:53.786063 11205 leader_election.cc:288] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:26:53.786208 11212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:53.787779 11212 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a [term 1 LEADER]: Becoming Leader. State: Replica: 81df64d70ee74d6dbfb37064593c0b9a, State: Running, Role: LEADER
I20260811 00:26:53.789098 11212 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81df64d70ee74d6dbfb37064593c0b9a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39689 } }
I20260811 00:26:53.790516 11205 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:26:53.795797 11214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81df64d70ee74d6dbfb37064593c0b9a. Latest consensus state: current_term: 1 leader_uuid: "81df64d70ee74d6dbfb37064593c0b9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81df64d70ee74d6dbfb37064593c0b9a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39689 } } }
I20260811 00:26:53.796947 11214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:53.797454 11213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81df64d70ee74d6dbfb37064593c0b9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81df64d70ee74d6dbfb37064593c0b9a" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39689 } } }
I20260811 00:26:53.801926 11213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a [sys.catalog]: This master's current role is: LEADER
I20260811 00:26:53.805078 11219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:26:53.817971 11219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:26:53.842947 11219 catalog_manager.cc:1347] Generated new cluster ID: 51276523524c4f068b38fd12d7c40cd7
I20260811 00:26:53.843459 11219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:26:53.891782 11219 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:26:53.895548 11219 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:26:53.940625 11219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a: Generated new TSK 0
I20260811 00:26:53.942304 11219 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.1.keytab.
Entry for principal oryx/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.1.keytab.
I20260811 00:26:54.069330 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:39689
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.1.keytab
--principal=oryx/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:54.669790 11234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:54.670152 11234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:54.670300 11234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:54.679891 11234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:54.680071 11234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:54.680171 11234 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:26:54.680418 11234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:26:54.692935 11234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.1.keytab
--principal=oryx/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.120.62:39689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:54.694501 11234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:54.696143 11234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:54.712998 11240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:54.712944 11239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:54.713222 11242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:54.718791 11234 server_base.cc:1034] running on GCE node
I20260811 00:26:54.719677 11234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:54.721058 11234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:54.722385 11234 hybrid_clock.cc:648] HybridClock initialized: now 1786408014722309 us; error 53 us; skew 500 ppm
I20260811 00:26:54.722785 11234 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:54 dist-test-slave-fd7l krb5kdc[11105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408014, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:54.727057 11234 init.cc:377] Logged in from keytab as oryx/127.2.120.1@KRBTEST.COM (short username oryx)
I20260811 00:26:54.729262 11234 webserver.cc:460] Webserver started at http://127.2.120.1:37643/ using document root <none> and password file <none>
I20260811 00:26:54.730247 11234 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:54.730413 11234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:54.730808 11234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:54.734387 11234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "8869d022846f4355a865a32c91f61c79"
format_stamp: "Formatted at 2026-08-11 00:26:54 on dist-test-slave-fd7l"
server_key: "7135bb89a78b84128382c7f75eb151ed"
server_key_iv: "b955c052ef5d88a2cb0cf5b31f3632e7"
server_key_version: "encryptionkey@0"
I20260811 00:26:54.735394 11234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "8869d022846f4355a865a32c91f61c79"
format_stamp: "Formatted at 2026-08-11 00:26:54 on dist-test-slave-fd7l"
server_key: "7135bb89a78b84128382c7f75eb151ed"
server_key_iv: "b955c052ef5d88a2cb0cf5b31f3632e7"
server_key_version: "encryptionkey@0"
I20260811 00:26:54.742403 11234 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260811 00:26:54.747135 11249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:54.749815 11234 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 00:26:54.750155 11234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "8869d022846f4355a865a32c91f61c79"
format_stamp: "Formatted at 2026-08-11 00:26:54 on dist-test-slave-fd7l"
server_key: "7135bb89a78b84128382c7f75eb151ed"
server_key_iv: "b955c052ef5d88a2cb0cf5b31f3632e7"
server_key_version: "encryptionkey@0"
I20260811 00:26:54.750732 11234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:54.770807 11234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:54.776337 11234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:54.776747 11234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:54.780149 11234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:54.780294 11234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:54.780491 11234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:54.780632 11234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 11 00:26:54 dist-test-slave-fd7l krb5kdc[11105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408014, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM
I20260811 00:26:54.813668 11260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:54.780881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:46259 (local address 127.2.120.62:39689)
0811 00:26:54.781740 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:54.781747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:54.782992 (+ 1245us) server_negotiation.cc:407] Connection header received
0811 00:26:54.788306 (+ 5314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:54.788314 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:54.788448 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:54.788738 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:54.791030 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:54.792232 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:54.795418 (+ 3186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:54.795895 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:54.796211 (+ 316us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:54.803503 (+ 7292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:54.803512 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:54.803517 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:54.803565 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:54.807867 (+ 4302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:54.809777 (+ 1910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:54.809783 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:54.809786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:54.809978 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:54.810744 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:54.810749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:54.810753 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:54.811133 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:54.811293 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:54.812915 (+ 1622us) server_negotiation.cc:299] Negotiation successful
0811 00:26:54.813258 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":535,"threads_started":1}
I20260811 00:26:54.815780 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:54.781711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:39689 (local address 127.2.120.1:46259)
0811 00:26:54.782736 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:54.782804 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:54.787743 (+ 4939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:54.789065 (+ 1322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:54.789075 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:54.789996 (+ 921us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:54.790732 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:54.790756 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:54.792549 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:54.792557 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:54.795183 (+ 2626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:54.795207 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:54.796333 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:54.796346 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:54.796725 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:54.797790 (+ 1065us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:54.797816 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:54.802993 (+ 5177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:54.808158 (+ 5165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:54.808179 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:54.808212 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:54.809523 (+ 1311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:54.810133 (+ 610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:54.810142 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:54.810149 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:54.810548 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:54.811293 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:54.811310 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:54.812051 (+ 741us) client_negotiation.cc:769] Sending connection context
0811 00:26:54.812567 (+ 516us) client_negotiation.cc:240] Negotiation successful
0811 00:26:54.813499 (+ 932us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":394,"threads_started":1}
I20260811 00:26:54.848060 11324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:54.825631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:39689 (local address 127.2.120.1:51575)
0811 00:26:54.826765 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:54.826800 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:54.827076 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:54.827705 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:54.827713 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:54.828735 (+ 1022us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:54.829316 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:54.829339 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:54.831043 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:54.831053 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:54.833567 (+ 2514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:54.833608 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:54.835443 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:54.835457 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:54.835816 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:54.836939 (+ 1123us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:54.836971 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:54.839071 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:54.843136 (+ 4065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:54.843161 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:54.843174 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:54.843998 (+ 824us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:54.844583 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:54.844591 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:54.844596 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:54.844780 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:54.846026 (+ 1246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:54.846034 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:54.846374 (+ 340us) client_negotiation.cc:769] Sending connection context
0811 00:26:54.846851 (+ 477us) client_negotiation.cc:240] Negotiation successful
0811 00:26:54.847604 (+ 753us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"thread_start_us":571,"threads_started":1}
I20260811 00:26:54.848628 11260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:54.825768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:51575 (local address 127.2.120.62:39689)
0811 00:26:54.826079 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:54.826091 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:54.827008 (+ 917us) server_negotiation.cc:407] Connection header received
0811 00:26:54.827399 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:54.827404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:54.827502 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:54.827592 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:54.829602 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:54.830737 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:54.833967 (+ 3230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:54.834818 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:54.834984 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:54.839416 (+ 4432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:54.839426 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:54.839433 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:54.839491 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:54.842927 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:54.844209 (+ 1282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:54.844218 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:54.844221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:54.844409 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:54.845173 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:54.845180 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:54.845183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:54.845699 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:54.846235 (+ 536us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:54.847628 (+ 1393us) server_negotiation.cc:299] Negotiation successful
0811 00:26:54.848034 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260811 00:26:54.861001 11234 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:33821
I20260811 00:26:54.861011 11382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:33821 every 8 connection(s)
I20260811 00:26:54.862531 11234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:26:54.865672 11384 heartbeater.cc:347] Connected to a master server at 127.2.120.62:39689
I20260811 00:26:54.866003 11384 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:54.867190 11384 heartbeater.cc:511] Master 127.2.120.62:39689 requested a full tablet report, sending...
I20260811 00:26:54.867220 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 11234
I20260811 00:26:54.867439 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:26:54.868400 2528 external_mini_cluster.cc:1428] Setting key 5b1f91a38da1ae38a9a8eddd749b7bc7
I20260811 00:26:54.870637 11155 ts_manager.cc:194] Registered new tserver with Master: 8869d022846f4355a865a32c91f61c79 (127.2.120.1:33821)
I20260811 00:26:54.873132 11155 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:46259
WARNING: no policy specified for oryx/127.2.120.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.120.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.2.keytab.
Entry for principal oryx/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.2.keytab.
I20260811 00:26:54.937383 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:39689
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.2.keytab
--principal=oryx/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:26:55.326960 11390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:55.327363 11390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:55.327543 11390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:55.335945 11390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:55.336158 11390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:55.336336 11390 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:26:55.336546 11390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:26:55.347689 11390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.2.keytab
--principal=oryx/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.120.62:39689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:55.349385 11390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:55.351009 11390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:55.367936 11398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:55.368391 11396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:55.367919 11395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:55.368837 11390 server_base.cc:1034] running on GCE node
I20260811 00:26:55.369591 11390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:55.370747 11390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:55.372056 11390 hybrid_clock.cc:648] HybridClock initialized: now 1786408015371974 us; error 45 us; skew 500 ppm
I20260811 00:26:55.372421 11390 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:55 dist-test-slave-fd7l krb5kdc[11105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408015, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:55.376318 11390 init.cc:377] Logged in from keytab as oryx/127.2.120.2@KRBTEST.COM (short username oryx)
I20260811 00:26:55.378321 11390 webserver.cc:460] Webserver started at http://127.2.120.2:41397/ using document root <none> and password file <none>
I20260811 00:26:55.379140 11390 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:55.379294 11390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:55.379664 11390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:55.383251 11390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "97a00c1cd50a41f2af3d16254a462e87"
format_stamp: "Formatted at 2026-08-11 00:26:55 on dist-test-slave-fd7l"
server_key: "ba4919a2ac99659ec0e6b109a9665537"
server_key_iv: "a64c8a847748e1e91a20f594a27832d7"
server_key_version: "encryptionkey@0"
I20260811 00:26:55.384289 11390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "97a00c1cd50a41f2af3d16254a462e87"
format_stamp: "Formatted at 2026-08-11 00:26:55 on dist-test-slave-fd7l"
server_key: "ba4919a2ac99659ec0e6b109a9665537"
server_key_iv: "a64c8a847748e1e91a20f594a27832d7"
server_key_version: "encryptionkey@0"
I20260811 00:26:55.390903 11390 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:26:55.395982 11406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:55.398355 11390 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260811 00:26:55.398756 11390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "97a00c1cd50a41f2af3d16254a462e87"
format_stamp: "Formatted at 2026-08-11 00:26:55 on dist-test-slave-fd7l"
server_key: "ba4919a2ac99659ec0e6b109a9665537"
server_key_iv: "a64c8a847748e1e91a20f594a27832d7"
server_key_version: "encryptionkey@0"
I20260811 00:26:55.399094 11390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:55.422598 11390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:55.433969 11390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:55.434762 11390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:55.441896 11390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:55.442178 11390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:55.442481 11390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:55.442600 11390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:26:55 dist-test-slave-fd7l krb5kdc[11105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408015, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM
I20260811 00:26:55.493462 11417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:55.443754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:57759 (local address 127.2.120.62:39689)
0811 00:26:55.444733 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:55.444740 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:55.449006 (+ 4266us) server_negotiation.cc:407] Connection header received
0811 00:26:55.454697 (+ 5691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:55.454713 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:55.454952 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:55.455212 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:55.464202 (+ 8990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:55.466004 (+ 1802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:55.471988 (+ 5984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:55.473081 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:55.473269 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:55.482041 (+ 8772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:55.482053 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:55.482059 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:55.482148 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:55.486803 (+ 4655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:55.488654 (+ 1851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:55.488663 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:55.488667 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:55.488918 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:55.490028 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:55.490035 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:55.490041 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:55.490508 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:55.490691 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:55.492867 (+ 2176us) server_negotiation.cc:299] Negotiation successful
0811 00:26:55.493143 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":506,"threads_started":1}
I20260811 00:26:55.498016 11418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:55.445579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:39689 (local address 127.2.120.2:57759)
0811 00:26:55.448303 (+ 2724us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:55.448415 (+ 112us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:55.453462 (+ 5047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:55.456276 (+ 2814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:55.456315 (+ 39us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:55.460091 (+ 3776us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:55.463340 (+ 3249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:55.463442 (+ 102us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:55.466557 (+ 3115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:55.466571 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:55.471290 (+ 4719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:55.471374 (+ 84us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:55.473953 (+ 2579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:55.473965 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:55.474488 (+ 523us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:55.475899 (+ 1411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:55.475991 (+ 92us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:55.481186 (+ 5195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:55.487095 (+ 5909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:55.487117 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:55.487158 (+ 41us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:55.488332 (+ 1174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:55.489176 (+ 844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:55.489185 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:55.489190 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:55.489724 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:55.490727 (+ 1003us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:55.490745 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:55.491852 (+ 1107us) client_negotiation.cc:769] Sending connection context
0811 00:26:55.492718 (+ 866us) client_negotiation.cc:240] Negotiation successful
0811 00:26:55.494339 (+ 1621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2005,"thread_start_us":1246,"threads_started":1}
I20260811 00:26:55.512442 11390 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:42735
I20260811 00:26:55.512576 11540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:42735 every 8 connection(s)
I20260811 00:26:55.514520 11390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:26:55.517844 11541 heartbeater.cc:347] Connected to a master server at 127.2.120.62:39689
I20260811 00:26:55.518167 11541 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:55.519266 11541 heartbeater.cc:511] Master 127.2.120.62:39689 requested a full tablet report, sending...
I20260811 00:26:55.521437 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 11390
I20260811 00:26:55.521597 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:26:55.521787 11155 ts_manager.cc:194] Registered new tserver with Master: 97a00c1cd50a41f2af3d16254a462e87 (127.2.120.2:42735)
I20260811 00:26:55.522221 2528 external_mini_cluster.cc:1428] Setting key 9063338886b34fb4eacc9b23834c7f1d
I20260811 00:26:55.523051 11155 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:57759
I20260811 00:26:55.529332 11417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:55.508372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:58213 (local address 127.2.120.62:39689)
0811 00:26:55.508660 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:55.508669 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:55.513342 (+ 4673us) server_negotiation.cc:407] Connection header received
0811 00:26:55.513482 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:55.513487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:55.513583 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:55.513686 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:55.515131 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:55.516395 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:55.518365 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:55.519074 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:55.519272 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:55.522519 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:55.522527 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:55.522533 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:55.522584 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:55.526029 (+ 3445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:55.527092 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:55.527097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:55.527100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:55.527240 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:55.527677 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:55.527681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:55.527683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:55.528022 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:55.528210 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:55.528695 (+ 485us) server_negotiation.cc:299] Negotiation successful
0811 00:26:55.528955 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:26:55.529415 11532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:55.508277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:39689 (local address 127.2.120.2:58213)
0811 00:26:55.513091 (+ 4814us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:55.513127 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:55.513332 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:55.513760 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:55.513765 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:55.514465 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:55.514916 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:55.514932 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:55.516647 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:55.516652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:55.518145 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:55.518164 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:55.519475 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:55.519494 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:55.519652 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:55.520609 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:55.520650 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:55.522271 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:55.526197 (+ 3926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:55.526203 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:55.526208 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:55.526904 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:55.527373 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:55.527378 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:55.527382 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:55.527526 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:55.528170 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:55.528174 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:55.528364 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 00:26:55.528568 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 00:26:55.528943 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4619,"thread_start_us":4453,"threads_started":1}
WARNING: no policy specified for oryx/127.2.120.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.120.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.3.keytab.
Entry for principal oryx/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.3.keytab.
I20260811 00:26:55.593918 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:39689
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.3.keytab
--principal=oryx/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:26:55.877187 11384 heartbeater.cc:503] Master 127.2.120.62:39689 was elected leader, sending a full tablet report...
W20260811 00:26:55.982363 11548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:26:55.982801 11548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:26:55.983006 11548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:26:55.991860 11548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:26:55.992053 11548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:26:55.992233 11548 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:26:55.992408 11548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:26:56.003448 11548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.3.keytab
--principal=oryx/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.120.62:39689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:26:56.005056 11548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:26:56.006637 11548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:26:56.025638 11557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:56.025625 11554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:26:56.025668 11553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:26:56.026618 11548 server_base.cc:1034] running on GCE node
I20260811 00:26:56.027412 11548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:26:56.029002 11548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:26:56.030550 11548 hybrid_clock.cc:648] HybridClock initialized: now 1786408016030440 us; error 79 us; skew 500 ppm
I20260811 00:26:56.030992 11548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:26:56 dist-test-slave-fd7l krb5kdc[11105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408016, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:26:56.035199 11548 init.cc:377] Logged in from keytab as oryx/127.2.120.3@KRBTEST.COM (short username oryx)
I20260811 00:26:56.037254 11548 webserver.cc:460] Webserver started at http://127.2.120.3:40909/ using document root <none> and password file <none>
I20260811 00:26:56.038095 11548 fs_manager.cc:359] Metadata directory not provided
I20260811 00:26:56.038210 11548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:26:56.038532 11548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:26:56.042068 11548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "654bed556a1f4ad995ce2ecdc5e89337"
format_stamp: "Formatted at 2026-08-11 00:26:56 on dist-test-slave-fd7l"
server_key: "36a15b22a81e9635fca4f6a043157d8f"
server_key_iv: "52d8d4481863402212f26a758724b67f"
server_key_version: "encryptionkey@0"
I20260811 00:26:56.043033 11548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "654bed556a1f4ad995ce2ecdc5e89337"
format_stamp: "Formatted at 2026-08-11 00:26:56 on dist-test-slave-fd7l"
server_key: "36a15b22a81e9635fca4f6a043157d8f"
server_key_iv: "52d8d4481863402212f26a758724b67f"
server_key_version: "encryptionkey@0"
I20260811 00:26:56.049747 11548 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 00:26:56.055614 11565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:56.058055 11548 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.001s
I20260811 00:26:56.058419 11548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "654bed556a1f4ad995ce2ecdc5e89337"
format_stamp: "Formatted at 2026-08-11 00:26:56 on dist-test-slave-fd7l"
server_key: "36a15b22a81e9635fca4f6a043157d8f"
server_key_iv: "52d8d4481863402212f26a758724b67f"
server_key_version: "encryptionkey@0"
I20260811 00:26:56.058766 11548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:26:56.076104 11548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:26:56.083518 11548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:26:56.084102 11548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:26:56.089180 11548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:26:56.089438 11548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:26:56.089717 11548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:26:56.089819 11548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:26:56 dist-test-slave-fd7l krb5kdc[11105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408016, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM
I20260811 00:26:56.124131 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:56.090649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:48321 (local address 127.2.120.62:39689)
0811 00:26:56.091422 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:56.091430 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:56.094074 (+ 2644us) server_negotiation.cc:407] Connection header received
0811 00:26:56.098749 (+ 4675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:56.098757 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:56.098913 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:56.099063 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:56.102295 (+ 3232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.103572 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.107515 (+ 3943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.108249 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.108430 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.115389 (+ 6959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:56.115397 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:56.115402 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:56.115447 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:56.119833 (+ 4386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:56.121191 (+ 1358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:56.121195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:56.121198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:56.121351 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:56.121934 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:56.121942 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:56.121951 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:56.122285 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:56.122472 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:56.123380 (+ 908us) server_negotiation.cc:299] Negotiation successful
0811 00:26:56.123650 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":361,"threads_started":1}
I20260811 00:26:56.126127 11575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:56.091911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:39689 (local address 127.2.120.3:48321)
0811 00:26:56.093598 (+ 1687us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:56.093718 (+ 120us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:56.098241 (+ 4523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:56.099433 (+ 1192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:56.099458 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:56.100670 (+ 1212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:56.101947 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:56.102012 (+ 65us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.103883 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.103894 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:56.107198 (+ 3304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:56.107237 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.108831 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.108843 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:56.109320 (+ 477us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.110449 (+ 1129us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:56.110503 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:56.114976 (+ 4473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:56.120159 (+ 5183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:56.120180 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:56.120216 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:56.121008 (+ 792us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:56.121501 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:56.121508 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:56.121512 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:56.121779 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:56.122445 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:56.122455 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:56.122924 (+ 469us) client_negotiation.cc:769] Sending connection context
0811 00:26:56.123263 (+ 339us) client_negotiation.cc:240] Negotiation successful
0811 00:26:56.123936 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1037,"thread_start_us":647,"threads_started":1}
I20260811 00:26:56.150884 11669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:56.134150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:39689 (local address 127.2.120.3:41231)
0811 00:26:56.134924 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:56.134946 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:56.135185 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:56.135619 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:56.135624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:56.136250 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:56.136678 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:56.136692 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.138021 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.138026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:56.139972 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:56.139993 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.141131 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.141139 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:56.141320 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.142089 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:56.142110 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:56.143028 (+ 918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:56.147415 (+ 4387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:56.147422 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:56.147427 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:56.147919 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:56.148386 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:56.148392 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:56.148396 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:56.148637 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:56.149425 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:56.149433 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:56.149784 (+ 351us) client_negotiation.cc:769] Sending connection context
0811 00:26:56.150027 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 00:26:56.150388 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":456,"threads_started":1}
I20260811 00:26:56.151834 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:56.134045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:41231 (local address 127.2.120.62:39689)
0811 00:26:56.134258 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:56.134262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:56.135073 (+ 811us) server_negotiation.cc:407] Connection header received
0811 00:26:56.135359 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:56.135365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:56.135449 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:56.135576 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:56.136851 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.137837 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.140148 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.140899 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.141062 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.143285 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:56.143292 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:56.143296 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:56.143332 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:56.147219 (+ 3887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:56.148076 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:56.148084 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:56.148088 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:56.148252 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:56.148771 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:56.148780 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:56.148783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:56.149251 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:56.149485 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:56.151357 (+ 1872us) server_negotiation.cc:299] Negotiation successful
0811 00:26:56.151576 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 00:26:56.153187 11548 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:41297
I20260811 00:26:56.153299 11699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:41297 every 8 connection(s)
I20260811 00:26:56.154662 11548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:26:56.157038 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 11548
I20260811 00:26:56.157248 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:26:56.157622 11700 heartbeater.cc:347] Connected to a master server at 127.2.120.62:39689
I20260811 00:26:56.157859 2528 external_mini_cluster.cc:1428] Setting key 1c8b71088234bc1fd68edc8a693f57a5
I20260811 00:26:56.157972 11700 heartbeater.cc:464] Registering TS with master...
I20260811 00:26:56.158762 11700 heartbeater.cc:511] Master 127.2.120.62:39689 requested a full tablet report, sending...
I20260811 00:26:56.160413 11154 ts_manager.cc:194] Registered new tserver with Master: 654bed556a1f4ad995ce2ecdc5e89337 (127.2.120.3:41297)
I20260811 00:26:56.161593 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:26:56.161621 11154 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:48321
Aug 11 00:26:56 dist-test-slave-fd7l krb5kdc[11105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408016, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 00:26:56 dist-test-slave-fd7l krb5kdc[11105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:26:56 dist-test-slave-fd7l krb5kdc[11105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 00:26:56.228950 11574 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:26:56.215774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57266 (local address 127.2.120.62:39689)
0811 00:26:56.216009 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:56.216020 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:56.216095 (+ 75us) server_negotiation.cc:407] Connection header received
0811 00:26:56.216362 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:56.216366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:56.216441 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:56.216601 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:56.218177 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.219101 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.220791 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.221420 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.221600 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.228461 (+ 6861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57266: BlockingRecv error: recv got EOF from 127.0.0.1:57266 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 00:26:56 dist-test-slave-fd7l krb5kdc[11105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408016, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM
I20260811 00:26:56.251746 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:56.233436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57268 (local address 127.2.120.62:39689)
0811 00:26:56.233731 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:56.233739 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:56.234012 (+ 273us) server_negotiation.cc:407] Connection header received
0811 00:26:56.234162 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:56.234167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:56.234245 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:56.234344 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:56.236191 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.237172 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.239074 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.239579 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.239730 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.245098 (+ 5368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:56.245105 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:56.245109 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:56.245156 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:56.248251 (+ 3095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:56.249245 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:56.249253 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:56.249257 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:56.249416 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:56.249911 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:56.249919 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:56.249923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:56.250269 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:56.250601 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:56.251047 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 00:26:56.251337 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 00:26:56.257704 11154 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57268:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:26:56.266345 11154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:26:56.311604 11721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:56.284668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:42735 (local address 127.0.0.1:46684)
0811 00:26:56.298613 (+ 13945us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:56.298704 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:56.299058 (+ 354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:56.300780 (+ 1722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:56.300789 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:56.300848 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:56.302325 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:56.302373 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.304301 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.304311 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:56.307796 (+ 3485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:56.307816 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.310190 (+ 2374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.310197 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:56.310431 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.310497 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 00:26:56.310657 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 00:26:56.310938 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13422,"mutex_wait_us":1088,"spinlock_wait_cycles":7168,"thread_start_us":14132,"threads_started":1}
I20260811 00:26:56.311883 11722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:56.284874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46684 (local address 127.2.120.2:42735)
0811 00:26:56.298598 (+ 13724us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:56.298609 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:56.299684 (+ 1075us) server_negotiation.cc:407] Connection header received
0811 00:26:56.299801 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:56.299809 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:56.300051 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:56.300331 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:56.302585 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.304131 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.308042 (+ 3911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.309615 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.309764 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.310016 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:56.310974 (+ 958us) server_negotiation.cc:299] Negotiation successful
0811 00:26:56.311500 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13555,"thread_start_us":13402,"threads_started":1}
I20260811 00:26:56.312153 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:56.301052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:33821 (local address 127.0.0.1:53720)
0811 00:26:56.302232 (+ 1180us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:56.302259 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:56.302429 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:56.303422 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:56.303427 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:56.303453 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:56.303934 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:56.303950 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.305731 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.305738 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:56.308152 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:56.308170 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.310759 (+ 2589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.310772 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:56.311062 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.311117 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 00:26:56.311517 (+ 400us) client_negotiation.cc:240] Negotiation successful
0811 00:26:56.311720 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":966,"thread_start_us":733,"threads_started":1}
I20260811 00:26:56.312352 11724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:56.285408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48550 (local address 127.2.120.3:41297)
0811 00:26:56.298002 (+ 12594us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:56.298021 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:56.299329 (+ 1308us) server_negotiation.cc:407] Connection header received
0811 00:26:56.299657 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:56.299668 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:56.300037 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:56.300330 (+ 293us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:56.301854 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.303783 (+ 1929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.308043 (+ 4260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.310358 (+ 2315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.310484 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.310733 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:56.311784 (+ 1051us) server_negotiation.cc:299] Negotiation successful
0811 00:26:56.311999 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12105,"thread_start_us":13206,"threads_started":1}
I20260811 00:26:56.312928 11720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:56.284866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:41297 (local address 127.0.0.1:48550)
0811 00:26:56.299143 (+ 14277us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:56.299161 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:56.299380 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:56.300780 (+ 1400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:56.300789 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:56.300840 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:56.301486 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:56.301509 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.304566 (+ 3057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.304570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:56.307796 (+ 3226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:56.307816 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.310978 (+ 3162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.310984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:56.311187 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.311224 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 00:26:56.311315 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 00:26:56.311467 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14134,"mutex_wait_us":274,"spinlock_wait_cycles":768,"thread_start_us":14448,"threads_started":1}
I20260811 00:26:56.313143 11725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:56.300369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53720 (local address 127.2.120.1:33821)
0811 00:26:56.301674 (+ 1305us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:56.301686 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:56.302412 (+ 726us) server_negotiation.cc:407] Connection header received
0811 00:26:56.302659 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:56.302669 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:56.302962 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:56.303196 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:56.304142 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.305562 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.308354 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.309867 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.310014 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.310358 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:56.312479 (+ 2121us) server_negotiation.cc:299] Negotiation successful
0811 00:26:56.312759 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1109,"thread_start_us":958,"threads_started":1}
I20260811 00:26:56.316939 11622 tablet_service.cc:1467] Processing CreateTablet for tablet 669d9a25ec5e4fb68e4c4d57997d5f14 (DEFAULT_TABLE table=test-table [id=7d1754fe9bdd4fac8e0c6b33d07e2b48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:56.317265 11463 tablet_service.cc:1467] Processing CreateTablet for tablet 669d9a25ec5e4fb68e4c4d57997d5f14 (DEFAULT_TABLE table=test-table [id=7d1754fe9bdd4fac8e0c6b33d07e2b48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:56.317265 11307 tablet_service.cc:1467] Processing CreateTablet for tablet 669d9a25ec5e4fb68e4c4d57997d5f14 (DEFAULT_TABLE table=test-table [id=7d1754fe9bdd4fac8e0c6b33d07e2b48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:56.318472 11622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 669d9a25ec5e4fb68e4c4d57997d5f14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:56.318754 11307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 669d9a25ec5e4fb68e4c4d57997d5f14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:56.318781 11463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 669d9a25ec5e4fb68e4c4d57997d5f14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:56.333313 11728 tablet_bootstrap.cc:492] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337: Bootstrap starting.
I20260811 00:26:56.337414 11730 tablet_bootstrap.cc:492] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79: Bootstrap starting.
I20260811 00:26:56.338574 11728 tablet_bootstrap.cc:654] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:56.340313 11728 log.cc:824] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:56.341830 11730 tablet_bootstrap.cc:654] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:56.343272 11730 log.cc:824] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:56.343357 11729 tablet_bootstrap.cc:492] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87: Bootstrap starting.
I20260811 00:26:56.345112 11728 tablet_bootstrap.cc:492] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337: No bootstrap required, opened a new log
I20260811 00:26:56.345686 11728 ts_tablet_manager.cc:1397] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.002s
I20260811 00:26:56.347312 11729 tablet_bootstrap.cc:654] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:56.348917 11729 log.cc:824] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87: Log is configured to *not* fsync() on all Append() calls
I20260811 00:26:56.350968 11730 tablet_bootstrap.cc:492] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79: No bootstrap required, opened a new log
I20260811 00:26:56.351338 11730 ts_tablet_manager.cc:1397] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260811 00:26:56.352161 11729 tablet_bootstrap.cc:492] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87: No bootstrap required, opened a new log
I20260811 00:26:56.352469 11729 ts_tablet_manager.cc:1397] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 00:26:56.357358 11728 raft_consensus.cc:348] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } }
I20260811 00:26:56.357944 11728 raft_consensus.cc:374] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:56.358093 11728 raft_consensus.cc:729] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 654bed556a1f4ad995ce2ecdc5e89337, State: Initialized, Role: FOLLOWER
I20260811 00:26:56.358896 11728 consensus_queue.cc:260] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } }
I20260811 00:26:56.360306 11700 heartbeater.cc:503] Master 127.2.120.62:39689 was elected leader, sending a full tablet report...
I20260811 00:26:56.361143 11728 ts_tablet_manager.cc:1428] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337: Time spent starting tablet: real 0.015s user 0.007s sys 0.008s
I20260811 00:26:56.363406 11729 raft_consensus.cc:348] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } }
I20260811 00:26:56.363786 11730 raft_consensus.cc:348] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } }
I20260811 00:26:56.364097 11729 raft_consensus.cc:374] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:56.364261 11730 raft_consensus.cc:374] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:56.364239 11729 raft_consensus.cc:729] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97a00c1cd50a41f2af3d16254a462e87, State: Initialized, Role: FOLLOWER
I20260811 00:26:56.364372 11730 raft_consensus.cc:729] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8869d022846f4355a865a32c91f61c79, State: Initialized, Role: FOLLOWER
I20260811 00:26:56.365092 11729 consensus_queue.cc:260] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } }
I20260811 00:26:56.365221 11730 consensus_queue.cc:260] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } }
I20260811 00:26:56.366698 11541 heartbeater.cc:503] Master 127.2.120.62:39689 was elected leader, sending a full tablet report...
I20260811 00:26:56.367794 11730 ts_tablet_manager.cc:1428] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260811 00:26:56.369310 11729 ts_tablet_manager.cc:1428] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87: Time spent starting tablet: real 0.017s user 0.014s sys 0.000s
I20260811 00:26:56.402813 11735 raft_consensus.cc:482] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:26:56.403522 11735 raft_consensus.cc:504] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } }
I20260811 00:26:56.405849 11735 leader_election.cc:288] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97a00c1cd50a41f2af3d16254a462e87 (127.2.120.2:42735), 8869d022846f4355a865a32c91f61c79 (127.2.120.1:33821)
W20260811 00:26:56.408991 11701 tablet.cc:2369] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:26:56.413565 11575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:56.406068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:42735 (local address 127.2.120.3:52581)
0811 00:26:56.406325 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:56.406348 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:56.406510 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:56.407293 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:56.407297 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:56.407331 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:56.407548 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:56.407557 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.408617 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.408623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:56.410561 (+ 1938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:56.410580 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.412742 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.412753 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:56.412999 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.413048 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 00:26:56.413149 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 00:26:56.413270 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":62}
I20260811 00:26:56.413689 11722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:56.406654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:52581 (local address 127.2.120.2:42735)
0811 00:26:56.406875 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:56.406883 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:56.406917 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:26:56.406995 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:56.407000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:56.407084 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:56.407183 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:56.407716 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.408427 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.410888 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.412190 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.412747 (+ 557us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.412890 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:56.413217 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 00:26:56.413397 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:26:56.414793 11725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:56.406926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:45137 (local address 127.2.120.1:33821)
0811 00:26:56.407120 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:56.407126 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:56.407336 (+ 210us) server_negotiation.cc:407] Connection header received
0811 00:26:56.407619 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:56.407623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:56.407677 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:56.407785 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:56.408336 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.409370 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.411594 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.413350 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.413522 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.413633 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:56.414199 (+ 566us) server_negotiation.cc:299] Negotiation successful
0811 00:26:56.414427 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 00:26:56.414911 11738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:56.406362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:33821 (local address 127.2.120.3:45137)
0811 00:26:56.407238 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:56.407259 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:56.407440 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:56.407802 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:56.407806 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:56.407831 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:56.408158 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:56.408168 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.409542 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.409548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:56.411340 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:56.411351 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.413657 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.413666 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:56.413854 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.413900 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 00:26:56.414020 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 00:26:56.414446 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":3,"thread_start_us":726,"threads_started":1}
I20260811 00:26:56.415122 11485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "669d9a25ec5e4fb68e4c4d57997d5f14" candidate_uuid: "654bed556a1f4ad995ce2ecdc5e89337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97a00c1cd50a41f2af3d16254a462e87" is_pre_election: true
I20260811 00:26:56.415741 11485 raft_consensus.cc:2454] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 654bed556a1f4ad995ce2ecdc5e89337 in term 0.
I20260811 00:26:56.416424 11334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "669d9a25ec5e4fb68e4c4d57997d5f14" candidate_uuid: "654bed556a1f4ad995ce2ecdc5e89337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8869d022846f4355a865a32c91f61c79" is_pre_election: true
I20260811 00:26:56.416782 11566 leader_election.cc:302] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 654bed556a1f4ad995ce2ecdc5e89337, 97a00c1cd50a41f2af3d16254a462e87; no voters:
I20260811 00:26:56.417088 11334 raft_consensus.cc:2454] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 654bed556a1f4ad995ce2ecdc5e89337 in term 0.
I20260811 00:26:56.417322 11735 raft_consensus.cc:2781] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:26:56.417482 11735 raft_consensus.cc:482] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:26:56.417623 11735 raft_consensus.cc:3037] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:56.420343 11735 raft_consensus.cc:504] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } }
I20260811 00:26:56.421119 11735 leader_election.cc:288] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [CANDIDATE]: Term 1 election: Requested vote from peers 97a00c1cd50a41f2af3d16254a462e87 (127.2.120.2:42735), 8869d022846f4355a865a32c91f61c79 (127.2.120.1:33821)
I20260811 00:26:56.421974 11334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "669d9a25ec5e4fb68e4c4d57997d5f14" candidate_uuid: "654bed556a1f4ad995ce2ecdc5e89337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8869d022846f4355a865a32c91f61c79"
I20260811 00:26:56.421994 11485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "669d9a25ec5e4fb68e4c4d57997d5f14" candidate_uuid: "654bed556a1f4ad995ce2ecdc5e89337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97a00c1cd50a41f2af3d16254a462e87"
I20260811 00:26:56.422276 11485 raft_consensus.cc:3037] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:56.422489 11334 raft_consensus.cc:3037] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:56.424886 11485 raft_consensus.cc:2454] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 654bed556a1f4ad995ce2ecdc5e89337 in term 1.
I20260811 00:26:56.424943 11334 raft_consensus.cc:2454] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 654bed556a1f4ad995ce2ecdc5e89337 in term 1.
I20260811 00:26:56.425490 11566 leader_election.cc:302] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 654bed556a1f4ad995ce2ecdc5e89337, 97a00c1cd50a41f2af3d16254a462e87; no voters:
I20260811 00:26:56.425912 11735 raft_consensus.cc:2781] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:56.426390 11735 raft_consensus.cc:686] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 LEADER]: Becoming Leader. State: Replica: 654bed556a1f4ad995ce2ecdc5e89337, State: Running, Role: LEADER
I20260811 00:26:56.426963 11735 consensus_queue.cc:237] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } }
I20260811 00:26:56.432910 11154 catalog_manager.cc:5495] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 reported cstate change: term changed from 0 to 1, leader changed from <none> to 654bed556a1f4ad995ce2ecdc5e89337 (127.2.120.3). New cstate: current_term: 1 leader_uuid: "654bed556a1f4ad995ce2ecdc5e89337" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } health_report { overall_health: UNKNOWN } } }
I20260811 00:26:56.479233 11724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:56.471403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48560 (local address 127.2.120.3:41297)
0811 00:26:56.471941 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:56.471947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:56.471997 (+ 50us) server_negotiation.cc:407] Connection header received
0811 00:26:56.472108 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:56.472113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:56.472202 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:56.472362 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:26:56.472977 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.474207 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.475896 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.476491 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.476785 (+ 294us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.477781 (+ 996us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:26:56.478482 (+ 701us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:26:56.478678 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:56.478885 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 00:26:56.478985 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260811 00:26:56.490190 11334 raft_consensus.cc:1261] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79 [term 1 FOLLOWER]: Refusing update from remote peer 654bed556a1f4ad995ce2ecdc5e89337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:56.490190 11485 raft_consensus.cc:1261] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87 [term 1 FOLLOWER]: Refusing update from remote peer 654bed556a1f4ad995ce2ecdc5e89337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:56.491787 11735 consensus_queue.cc:1035] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:56.492208 11739 consensus_queue.cc:1035] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:56.540045 11153 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:56.540405 11153 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:56.545775 11153 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a: Sending DeleteTablet for 3 replicas of tablet 669d9a25ec5e4fb68e4c4d57997d5f14
I20260811 00:26:56.548247 11463 tablet_service.cc:1514] Processing DeleteTablet for tablet 669d9a25ec5e4fb68e4c4d57997d5f14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:56 UTC) from {username='oryx', principal='oryx/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:46684
I20260811 00:26:56.548236 11622 tablet_service.cc:1514] Processing DeleteTablet for tablet 669d9a25ec5e4fb68e4c4d57997d5f14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:56 UTC) from {username='oryx', principal='oryx/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:48550
I20260811 00:26:56.548717 11307 tablet_service.cc:1514] Processing DeleteTablet for tablet 669d9a25ec5e4fb68e4c4d57997d5f14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:56 UTC) from {username='oryx', principal='oryx/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:53720
I20260811 00:26:56.550011 11756 tablet_replica.cc:330] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79: stopping tablet replica
I20260811 00:26:56.550050 11153 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57268:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 00:26:56.550669 11756 raft_consensus.cc:2229] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:56.551441 11756 raft_consensus.cc:2258] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 00:26:56.551574 11153 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:26:56.554292 11756 ts_tablet_manager.cc:1905] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:56.562389 11307 tablet_service.cc:1467] Processing CreateTablet for tablet 707b2c1a462e463b94f1dd8092b8a795 (DEFAULT_TABLE table=test-table [id=6794d766b6cc4eed86b9bd37faec8bda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:56.562937 11307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707b2c1a462e463b94f1dd8092b8a795. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:56.563015 11621 tablet_service.cc:1467] Processing CreateTablet for tablet 707b2c1a462e463b94f1dd8092b8a795 (DEFAULT_TABLE table=test-table [id=6794d766b6cc4eed86b9bd37faec8bda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:56.563514 11621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707b2c1a462e463b94f1dd8092b8a795. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:56.564925 11756 ts_tablet_manager.cc:1918] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:26:56.565161 11756 log.cc:1196] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/669d9a25ec5e4fb68e4c4d57997d5f14
I20260811 00:26:56.565968 11756 ts_tablet_manager.cc:1939] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 8869d022846f4355a865a32c91f61c79: Deleting consensus metadata
I20260811 00:26:56.568821 11138 catalog_manager.cc:4841] TS 8869d022846f4355a865a32c91f61c79 (127.2.120.1:33821): tablet 669d9a25ec5e4fb68e4c4d57997d5f14 (table test-table [id=7d1754fe9bdd4fac8e0c6b33d07e2b48]) successfully deleted
I20260811 00:26:56.569162 11462 tablet_service.cc:1467] Processing CreateTablet for tablet 707b2c1a462e463b94f1dd8092b8a795 (DEFAULT_TABLE table=test-table [id=6794d766b6cc4eed86b9bd37faec8bda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:56.569821 11462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707b2c1a462e463b94f1dd8092b8a795. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:56.570895 11730 tablet_bootstrap.cc:492] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79: Bootstrap starting.
I20260811 00:26:56.573372 11730 tablet_bootstrap.cc:654] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:56.579375 11728 tablet_bootstrap.cc:492] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337: Bootstrap starting.
I20260811 00:26:56.582386 11729 tablet_bootstrap.cc:492] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87: Bootstrap starting.
I20260811 00:26:56.583895 11730 tablet_bootstrap.cc:492] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79: No bootstrap required, opened a new log
I20260811 00:26:56.583890 11754 tablet_replica.cc:330] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87: stopping tablet replica
I20260811 00:26:56.584085 11730 ts_tablet_manager.cc:1397] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.005s
I20260811 00:26:56.584441 11754 raft_consensus.cc:2229] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:56.585023 11754 raft_consensus.cc:2258] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:56.585023 11730 raft_consensus.cc:348] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:56.585280 11730 raft_consensus.cc:374] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:56.585390 11730 raft_consensus.cc:729] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8869d022846f4355a865a32c91f61c79, State: Initialized, Role: FOLLOWER
I20260811 00:26:56.585913 11730 consensus_queue.cc:260] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:56.586575 11730 ts_tablet_manager.cc:1428] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 00:26:56.587855 11729 tablet_bootstrap.cc:654] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:56.588284 11728 tablet_bootstrap.cc:654] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:56.590224 11755 tablet_replica.cc:330] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337: stopping tablet replica
I20260811 00:26:56.590930 11755 raft_consensus.cc:2229] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:26:56.592106 11755 raft_consensus.cc:2258] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:56.595224 11755 ts_tablet_manager.cc:1905] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:56.596928 11754 ts_tablet_manager.cc:1905] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:56.605363 11755 ts_tablet_manager.cc:1918] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:26:56.605602 11728 tablet_bootstrap.cc:492] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337: No bootstrap required, opened a new log
I20260811 00:26:56.605615 11755 log.cc:1196] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/669d9a25ec5e4fb68e4c4d57997d5f14
I20260811 00:26:56.605794 11728 ts_tablet_manager.cc:1397] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337: Time spent bootstrapping tablet: real 0.027s user 0.006s sys 0.004s
I20260811 00:26:56.606208 11729 tablet_bootstrap.cc:492] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87: No bootstrap required, opened a new log
I20260811 00:26:56.606297 11755 ts_tablet_manager.cc:1939] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 654bed556a1f4ad995ce2ecdc5e89337: Deleting consensus metadata
I20260811 00:26:56.606421 11729 ts_tablet_manager.cc:1397] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.004s
I20260811 00:26:56.606840 11728 raft_consensus.cc:348] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:56.607095 11728 raft_consensus.cc:374] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:56.607242 11728 raft_consensus.cc:729] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 654bed556a1f4ad995ce2ecdc5e89337, State: Initialized, Role: FOLLOWER
I20260811 00:26:56.607367 11729 raft_consensus.cc:348] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:56.607661 11729 raft_consensus.cc:374] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:56.607764 11729 raft_consensus.cc:729] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97a00c1cd50a41f2af3d16254a462e87, State: Initialized, Role: FOLLOWER
I20260811 00:26:56.607668 11728 consensus_queue.cc:260] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:56.608067 11729 consensus_queue.cc:260] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:56.608290 11138 catalog_manager.cc:4841] TS 654bed556a1f4ad995ce2ecdc5e89337 (127.2.120.3:41297): tablet 669d9a25ec5e4fb68e4c4d57997d5f14 (table test-table [id=7d1754fe9bdd4fac8e0c6b33d07e2b48]) successfully deleted
I20260811 00:26:56.609179 11728 ts_tablet_manager.cc:1428] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 00:26:56.611697 11729 ts_tablet_manager.cc:1428] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 00:26:56.613677 11754 ts_tablet_manager.cc:1918] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:26:56.613865 11754 log.cc:1196] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/669d9a25ec5e4fb68e4c4d57997d5f14
I20260811 00:26:56.614461 11754 ts_tablet_manager.cc:1939] T 669d9a25ec5e4fb68e4c4d57997d5f14 P 97a00c1cd50a41f2af3d16254a462e87: Deleting consensus metadata
I20260811 00:26:56.616686 11136 catalog_manager.cc:4841] TS 97a00c1cd50a41f2af3d16254a462e87 (127.2.120.2:42735): tablet 669d9a25ec5e4fb68e4c4d57997d5f14 (table test-table [id=7d1754fe9bdd4fac8e0c6b33d07e2b48]) successfully deleted
W20260811 00:26:56.616842 11385 tablet.cc:2369] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:26:56.769702 11543 tablet.cc:2369] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:26:56.796231 11752 raft_consensus.cc:482] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:26:56.796559 11752 raft_consensus.cc:504] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:56.797446 11752 leader_election.cc:288] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8869d022846f4355a865a32c91f61c79 (127.2.120.1:33821), 97a00c1cd50a41f2af3d16254a462e87 (127.2.120.2:42735)
I20260811 00:26:56.798151 11485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707b2c1a462e463b94f1dd8092b8a795" candidate_uuid: "654bed556a1f4ad995ce2ecdc5e89337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97a00c1cd50a41f2af3d16254a462e87" is_pre_election: true
I20260811 00:26:56.798138 11334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707b2c1a462e463b94f1dd8092b8a795" candidate_uuid: "654bed556a1f4ad995ce2ecdc5e89337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8869d022846f4355a865a32c91f61c79" is_pre_election: true
I20260811 00:26:56.798444 11334 raft_consensus.cc:2454] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 654bed556a1f4ad995ce2ecdc5e89337 in term 0.
I20260811 00:26:56.798444 11485 raft_consensus.cc:2454] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 654bed556a1f4ad995ce2ecdc5e89337 in term 0.
I20260811 00:26:56.798979 11566 leader_election.cc:302] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 654bed556a1f4ad995ce2ecdc5e89337, 97a00c1cd50a41f2af3d16254a462e87; no voters:
I20260811 00:26:56.799361 11752 raft_consensus.cc:2781] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:26:56.799495 11752 raft_consensus.cc:482] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:26:56.799587 11752 raft_consensus.cc:3037] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:56.802856 11752 raft_consensus.cc:504] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:56.803721 11752 leader_election.cc:288] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [CANDIDATE]: Term 1 election: Requested vote from peers 8869d022846f4355a865a32c91f61c79 (127.2.120.1:33821), 97a00c1cd50a41f2af3d16254a462e87 (127.2.120.2:42735)
I20260811 00:26:56.804353 11485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707b2c1a462e463b94f1dd8092b8a795" candidate_uuid: "654bed556a1f4ad995ce2ecdc5e89337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97a00c1cd50a41f2af3d16254a462e87"
I20260811 00:26:56.804407 11334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707b2c1a462e463b94f1dd8092b8a795" candidate_uuid: "654bed556a1f4ad995ce2ecdc5e89337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8869d022846f4355a865a32c91f61c79"
I20260811 00:26:56.804658 11485 raft_consensus.cc:3037] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:56.804694 11334 raft_consensus.cc:3037] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:56.807257 11334 raft_consensus.cc:2454] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 654bed556a1f4ad995ce2ecdc5e89337 in term 1.
I20260811 00:26:56.807898 11567 leader_election.cc:302] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 654bed556a1f4ad995ce2ecdc5e89337, 8869d022846f4355a865a32c91f61c79; no voters:
I20260811 00:26:56.807996 11485 raft_consensus.cc:2454] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 654bed556a1f4ad995ce2ecdc5e89337 in term 1.
I20260811 00:26:56.808317 11752 raft_consensus.cc:2781] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:56.808607 11752 raft_consensus.cc:686] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 LEADER]: Becoming Leader. State: Replica: 654bed556a1f4ad995ce2ecdc5e89337, State: Running, Role: LEADER
I20260811 00:26:56.808944 11752 consensus_queue.cc:237] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:56.812207 11154 catalog_manager.cc:5495] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 reported cstate change: term changed from 0 to 1, leader changed from <none> to 654bed556a1f4ad995ce2ecdc5e89337 (127.2.120.3). New cstate: current_term: 1 leader_uuid: "654bed556a1f4ad995ce2ecdc5e89337" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } health_report { overall_health: HEALTHY } } }
Aug 11 00:26:56 dist-test-slave-fd7l krb5kdc[11105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408013, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.62@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM
I20260811 00:26:56.905596 11767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:56.867716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57282 (local address 127.2.120.62:39689)
0811 00:26:56.881245 (+ 13529us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:56.881253 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:56.881538 (+ 285us) server_negotiation.cc:407] Connection header received
0811 00:26:56.881788 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:56.881794 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:56.881882 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:56.882020 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:56.883613 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.884891 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.886922 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.887602 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.887757 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.895861 (+ 8104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:56.895872 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:56.895880 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:56.895927 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:56.900154 (+ 4227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:56.902022 (+ 1868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:56.902030 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:56.902052 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:56.902305 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:56.903024 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:56.903032 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:56.903066 (+ 34us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:56.903535 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:56.903722 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:56.904486 (+ 764us) server_negotiation.cc:299] Negotiation successful
0811 00:26:56.904881 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":13270,"thread_start_us":13065,"threads_started":1}
I20260811 00:26:56.905594 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:56.867720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:39689 (local address 127.0.0.1:57282)
0811 00:26:56.881406 (+ 13686us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:56.881428 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:56.881633 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:56.882085 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:56.882090 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:56.883104 (+ 1014us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:26:56.883389 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:56.883405 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.885104 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.885110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:56.886712 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:56.886730 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:56.887823 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:56.887829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:56.888025 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:56.892937 (+ 4912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:26:56.892964 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:26:56.895539 (+ 2575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:26:56.900478 (+ 4939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:56.900506 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:56.900782 (+ 276us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:56.901770 (+ 988us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:56.902612 (+ 842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:26:56.902621 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:26:56.902626 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:26:56.902839 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:26:56.903788 (+ 949us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:26:56.903799 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:26:56.904077 (+ 278us) client_negotiation.cc:769] Sending connection context
0811 00:26:56.904369 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 00:26:56.905039 (+ 670us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13505,"mutex_wait_us":237,"thread_start_us":13386,"threads_started":1}
I20260811 00:26:56.913991 11154 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:57282:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 00:26:56.915637 11154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:26:56.927024 11307 tablet_service.cc:1467] Processing CreateTablet for tablet 379639095ea64332b7a0f75e76c736d2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6236f729ff134bf6b9b2f4c0483cf2af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:26:56.927794 11307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 379639095ea64332b7a0f75e76c736d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:56.927945 11622 tablet_service.cc:1467] Processing CreateTablet for tablet 379639095ea64332b7a0f75e76c736d2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6236f729ff134bf6b9b2f4c0483cf2af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:26:56.928828 11622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 379639095ea64332b7a0f75e76c736d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:56.933822 11463 tablet_service.cc:1467] Processing CreateTablet for tablet 379639095ea64332b7a0f75e76c736d2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6236f729ff134bf6b9b2f4c0483cf2af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:26:56.934554 11463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 379639095ea64332b7a0f75e76c736d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:56.937076 11728 tablet_bootstrap.cc:492] T 379639095ea64332b7a0f75e76c736d2 P 654bed556a1f4ad995ce2ecdc5e89337: Bootstrap starting.
I20260811 00:26:56.938854 11728 tablet_bootstrap.cc:654] T 379639095ea64332b7a0f75e76c736d2 P 654bed556a1f4ad995ce2ecdc5e89337: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:56.944567 11730 tablet_bootstrap.cc:492] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79: Bootstrap starting.
I20260811 00:26:56.946692 11729 tablet_bootstrap.cc:492] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87: Bootstrap starting.
I20260811 00:26:56.946761 11730 tablet_bootstrap.cc:654] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:56.948273 11728 tablet_bootstrap.cc:492] T 379639095ea64332b7a0f75e76c736d2 P 654bed556a1f4ad995ce2ecdc5e89337: No bootstrap required, opened a new log
I20260811 00:26:56.948805 11729 tablet_bootstrap.cc:654] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:56.949002 11728 ts_tablet_manager.cc:1397] T 379639095ea64332b7a0f75e76c736d2 P 654bed556a1f4ad995ce2ecdc5e89337: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.005s
I20260811 00:26:56.949950 11728 raft_consensus.cc:348] T 379639095ea64332b7a0f75e76c736d2 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:56.950316 11728 raft_consensus.cc:374] T 379639095ea64332b7a0f75e76c736d2 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:56.950492 11728 raft_consensus.cc:729] T 379639095ea64332b7a0f75e76c736d2 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 654bed556a1f4ad995ce2ecdc5e89337, State: Initialized, Role: FOLLOWER
I20260811 00:26:56.950891 11728 consensus_queue.cc:260] T 379639095ea64332b7a0f75e76c736d2 P 654bed556a1f4ad995ce2ecdc5e89337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:56.951459 11752 tablet_replica.cc:439] T 379639095ea64332b7a0f75e76c736d2 P 654bed556a1f4ad995ce2ecdc5e89337: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } }
I20260811 00:26:56.951797 11752 tablet_replica.cc:442] T 379639095ea64332b7a0f75e76c736d2 P 654bed556a1f4ad995ce2ecdc5e89337: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:26:56.952306 11730 tablet_bootstrap.cc:492] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79: No bootstrap required, opened a new log
I20260811 00:26:56.952569 11730 ts_tablet_manager.cc:1397] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.002s
I20260811 00:26:56.953646 11730 raft_consensus.cc:348] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:56.955579 11730 raft_consensus.cc:374] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:56.955600 11728 ts_tablet_manager.cc:1428] T 379639095ea64332b7a0f75e76c736d2 P 654bed556a1f4ad995ce2ecdc5e89337: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260811 00:26:56.955713 11730 raft_consensus.cc:729] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8869d022846f4355a865a32c91f61c79, State: Initialized, Role: FOLLOWER
I20260811 00:26:56.956048 11730 consensus_queue.cc:260] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:56.956974 11730 ts_tablet_manager.cc:1428] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260811 00:26:56.956861 11737 tablet_replica.cc:439] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } }
I20260811 00:26:56.957288 11737 tablet_replica.cc:442] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:26:56.958810 11729 tablet_bootstrap.cc:492] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87: No bootstrap required, opened a new log
I20260811 00:26:56.959026 11729 ts_tablet_manager.cc:1397] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.007s
I20260811 00:26:56.959914 11729 raft_consensus.cc:348] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:56.960597 11729 raft_consensus.cc:374] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:56.960793 11729 raft_consensus.cc:729] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97a00c1cd50a41f2af3d16254a462e87, State: Initialized, Role: FOLLOWER
I20260811 00:26:56.961338 11729 consensus_queue.cc:260] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:56.962121 11729 ts_tablet_manager.cc:1428] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 00:26:56.962050 11736 tablet_replica.cc:439] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } }
I20260811 00:26:56.962496 11736 tablet_replica.cc:442] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:26:57.149259 11736 raft_consensus.cc:482] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:26:57.149508 11736 raft_consensus.cc:504] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.151101 11736 leader_election.cc:288] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8869d022846f4355a865a32c91f61c79 (127.2.120.1:33821), 654bed556a1f4ad995ce2ecdc5e89337 (127.2.120.3:41297)
I20260811 00:26:57.163496 11774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:57.151517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:33821 (local address 127.2.120.2:42561)
0811 00:26:57.152401 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:57.152418 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:57.152639 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:57.153330 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:57.153339 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:57.153379 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:57.153813 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:57.153831 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.156055 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.156065 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:57.159204 (+ 3139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:57.159240 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.162080 (+ 2840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.162091 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:57.162412 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:57.162478 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 00:26:57.162631 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 00:26:57.162874 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"thread_start_us":706,"threads_started":1}
I20260811 00:26:57.165016 11775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:57.151604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:42561 (local address 127.2.120.1:33821)
0811 00:26:57.152130 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:57.152137 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:57.152553 (+ 416us) server_negotiation.cc:407] Connection header received
0811 00:26:57.152842 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:57.152849 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:57.152929 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:57.153036 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:57.154076 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.155741 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.159576 (+ 3835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.161509 (+ 1933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.161726 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:57.161918 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:57.164249 (+ 2331us) server_negotiation.cc:299] Negotiation successful
0811 00:26:57.164537 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":282,"threads_started":1}
I20260811 00:26:57.166421 11334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "379639095ea64332b7a0f75e76c736d2" candidate_uuid: "97a00c1cd50a41f2af3d16254a462e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8869d022846f4355a865a32c91f61c79" is_pre_election: true
I20260811 00:26:57.166726 11334 raft_consensus.cc:2454] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97a00c1cd50a41f2af3d16254a462e87 in term 0.
I20260811 00:26:57.167323 11409 leader_election.cc:302] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8869d022846f4355a865a32c91f61c79, 97a00c1cd50a41f2af3d16254a462e87; no voters:
I20260811 00:26:57.167860 11736 raft_consensus.cc:2781] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:26:57.168012 11736 raft_consensus.cc:482] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:26:57.168115 11736 raft_consensus.cc:3037] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:57.170660 11736 raft_consensus.cc:504] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.171509 11736 leader_election.cc:288] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [CANDIDATE]: Term 1 election: Requested vote from peers 8869d022846f4355a865a32c91f61c79 (127.2.120.1:33821), 654bed556a1f4ad995ce2ecdc5e89337 (127.2.120.3:41297)
I20260811 00:26:57.172088 11334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "379639095ea64332b7a0f75e76c736d2" candidate_uuid: "97a00c1cd50a41f2af3d16254a462e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8869d022846f4355a865a32c91f61c79"
I20260811 00:26:57.172379 11334 raft_consensus.cc:3037] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:57.174247 11774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:57.152719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:41297 (local address 127.2.120.2:59935)
0811 00:26:57.163849 (+ 11130us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:57.163872 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:57.164026 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:57.167236 (+ 3210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:57.167242 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:57.167277 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:26:57.167492 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:57.167509 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.169180 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.169186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:57.171393 (+ 2207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:57.171414 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.173411 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.173418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:57.173633 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:57.173677 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 00:26:57.173750 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 00:26:57.173924 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10999,"thread_start_us":12791,"threads_started":1}
I20260811 00:26:57.174829 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:57.152928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:59935 (local address 127.2.120.3:41297)
0811 00:26:57.166727 (+ 13799us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:57.166734 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:57.166756 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:26:57.166868 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:57.166875 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:57.166981 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:57.167171 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:26:57.167736 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.168970 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.171706 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.173190 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.173975 (+ 785us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:57.174188 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:57.174278 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 00:26:57.174446 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13627,"thread_start_us":13507,"threads_started":1}
I20260811 00:26:57.175365 11334 raft_consensus.cc:2454] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97a00c1cd50a41f2af3d16254a462e87 in term 1.
I20260811 00:26:57.176074 11409 leader_election.cc:302] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8869d022846f4355a865a32c91f61c79, 97a00c1cd50a41f2af3d16254a462e87; no voters:
I20260811 00:26:57.175910 11643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "379639095ea64332b7a0f75e76c736d2" candidate_uuid: "97a00c1cd50a41f2af3d16254a462e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "654bed556a1f4ad995ce2ecdc5e89337" is_pre_election: true
I20260811 00:26:57.175948 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "379639095ea64332b7a0f75e76c736d2" candidate_uuid: "97a00c1cd50a41f2af3d16254a462e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "654bed556a1f4ad995ce2ecdc5e89337"
I20260811 00:26:57.176487 11736 raft_consensus.cc:2781] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:57.176450 11642 raft_consensus.cc:3037] T 379639095ea64332b7a0f75e76c736d2 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:57.177419 11736 raft_consensus.cc:686] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [term 1 LEADER]: Becoming Leader. State: Replica: 97a00c1cd50a41f2af3d16254a462e87, State: Running, Role: LEADER
I20260811 00:26:57.177925 11736 consensus_queue.cc:237] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.180176 11642 raft_consensus.cc:2454] T 379639095ea64332b7a0f75e76c736d2 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97a00c1cd50a41f2af3d16254a462e87 in term 1.
I20260811 00:26:57.182422 11778 tablet_replica.cc:439] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87: TxnStatusTablet state changed. Reason: New leader 97a00c1cd50a41f2af3d16254a462e87. Latest consensus state: current_term: 1 leader_uuid: "97a00c1cd50a41f2af3d16254a462e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } }
I20260811 00:26:57.182833 11778 tablet_replica.cc:442] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:26:57.183626 11780 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 00:26:57.183254 11154 catalog_manager.cc:5495] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97a00c1cd50a41f2af3d16254a462e87 (127.2.120.2). New cstate: current_term: 1 leader_uuid: "97a00c1cd50a41f2af3d16254a462e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } health_report { overall_health: UNKNOWN } } }
I20260811 00:26:57.183931 11780 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 00:26:57.214833 11752 consensus_queue.cc:1035] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:57.223080 11752 consensus_queue.cc:1035] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:57.252758 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:57.245263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46698 (local address 127.2.120.2:42735)
0811 00:26:57.245963 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:57.245967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:57.245989 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:26:57.246078 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:57.246081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:57.246180 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:57.246385 (+ 205us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:26:57.246907 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.247921 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.249963 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.250482 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.250608 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:57.251285 (+ 677us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:26:57.251964 (+ 679us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:26:57.252121 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:57.252264 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 00:26:57.252346 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":347,"threads_started":1}
I20260811 00:26:57.252799 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:57.245137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:42735 (local address 127.0.0.1:46698)
0811 00:26:57.245411 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:57.245433 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:57.245603 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:57.246436 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:57.246442 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:57.246457 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:26:57.246684 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:57.246698 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.248076 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.248082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:57.249760 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:57.249778 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.250676 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.250684 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:57.250893 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:57.250995 (+ 102us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:26:57.252091 (+ 1096us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:26:57.252117 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 00:26:57.252218 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 00:26:57.252409 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 00:26:57.258232 11642 raft_consensus.cc:1261] T 379639095ea64332b7a0f75e76c736d2 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 FOLLOWER]: Refusing update from remote peer 97a00c1cd50a41f2af3d16254a462e87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:57.258194 11334 raft_consensus.cc:1261] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79 [term 1 FOLLOWER]: Refusing update from remote peer 97a00c1cd50a41f2af3d16254a462e87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:57.259086 11778 consensus_queue.cc:1035] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:57.259865 11736 consensus_queue.cc:1035] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 00:26:57.270838 11752 tablet_replica.cc:439] T 379639095ea64332b7a0f75e76c736d2 P 654bed556a1f4ad995ce2ecdc5e89337: TxnStatusTablet state changed. Reason: New leader 97a00c1cd50a41f2af3d16254a462e87. Latest consensus state: current_term: 1 leader_uuid: "97a00c1cd50a41f2af3d16254a462e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } }
I20260811 00:26:57.271366 11752 tablet_replica.cc:442] T 379639095ea64332b7a0f75e76c736d2 P 654bed556a1f4ad995ce2ecdc5e89337: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:26:57.274966 11737 tablet_replica.cc:439] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79: TxnStatusTablet state changed. Reason: New leader 97a00c1cd50a41f2af3d16254a462e87. Latest consensus state: current_term: 1 leader_uuid: "97a00c1cd50a41f2af3d16254a462e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } }
I20260811 00:26:57.275338 11737 tablet_replica.cc:442] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:26:57.278420 11785 mvcc.cc:205] Tried to move back new op lower bound from 7317127238680506368 to 7317127238366547968. Current Snapshot: MvccSnapshot[applied={T|T < 7317127238680506368}]
I20260811 00:26:57.279424 11778 tablet_replica.cc:439] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97a00c1cd50a41f2af3d16254a462e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } }
I20260811 00:26:57.279852 11778 tablet_replica.cc:442] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:26:57.293323 11778 tablet_replica.cc:439] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97a00c1cd50a41f2af3d16254a462e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } }
I20260811 00:26:57.293762 11778 tablet_replica.cc:442] T 379639095ea64332b7a0f75e76c736d2 P 97a00c1cd50a41f2af3d16254a462e87: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:26:57.294675 11784 mvcc.cc:205] Tried to move back new op lower bound from 7317127238680506368 to 7317127238366547968. Current Snapshot: MvccSnapshot[applied={T|T < 7317127238680506368}]
I20260811 00:26:57.295324 11737 tablet_replica.cc:439] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97a00c1cd50a41f2af3d16254a462e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } }
I20260811 00:26:57.295719 11737 tablet_replica.cc:442] T 379639095ea64332b7a0f75e76c736d2 P 8869d022846f4355a865a32c91f61c79: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:26:57.300800 11752 tablet_replica.cc:439] T 379639095ea64332b7a0f75e76c736d2 P 654bed556a1f4ad995ce2ecdc5e89337: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97a00c1cd50a41f2af3d16254a462e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } } }
I20260811 00:26:57.301234 11752 tablet_replica.cc:442] T 379639095ea64332b7a0f75e76c736d2 P 654bed556a1f4ad995ce2ecdc5e89337: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:26:57.319566 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:57.311487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:48381 (local address 127.2.120.2:42735)
0811 00:26:57.311724 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:57.311729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:57.312278 (+ 549us) server_negotiation.cc:407] Connection header received
0811 00:26:57.312615 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:57.312621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:57.312717 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:57.312837 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:26:57.313442 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.314660 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.316410 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.317083 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.317587 (+ 504us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:57.318329 (+ 742us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:26:57.318495 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:26:57.318700 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:57.318960 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 00:26:57.319089 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 00:26:57.319643 11798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:57.311264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:42735 (local address 127.2.120.3:48381)
0811 00:26:57.312133 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:57.312151 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:57.312389 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:57.312873 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:57.312878 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:57.312907 (+ 29us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:26:57.313257 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:57.313267 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.314980 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.314987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:57.316192 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:57.316210 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.317654 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.317666 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:57.317927 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:57.318022 (+ 95us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:26:57.318775 (+ 753us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:26:57.318791 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 00:26:57.318928 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 00:26:57.319143 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":520,"threads_started":1}
I20260811 00:26:57.374404 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:57.365154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:45109 (local address 127.2.120.3:41297)
0811 00:26:57.365376 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:57.365380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:57.366302 (+ 922us) server_negotiation.cc:407] Connection header received
0811 00:26:57.366595 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:57.366602 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:57.366699 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:57.366774 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:26:57.368079 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.369478 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.371744 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.372602 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.372777 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:57.373375 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:26:57.373544 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:26:57.373670 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:57.373939 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 00:26:57.374050 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 00:26:57.374754 11806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:57.365219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:41297 (local address 127.2.120.2:45109)
0811 00:26:57.366165 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0811 00:26:57.366186 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:26:57.366362 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:26:57.367263 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:26:57.367268 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:26:57.367286 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:26:57.367803 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:57.367820 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.369758 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.369764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:57.371533 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:26:57.371552 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.372883 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.372888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:26:57.373048 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:57.373141 (+ 93us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:26:57.373740 (+ 599us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:26:57.373774 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 00:26:57.373881 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 00:26:57.374062 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":573,"threads_started":1}
I20260811 00:26:57.422842 11153 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:57.423213 11153 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:57.427456 11153 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a: Sending DeleteTablet for 3 replicas of tablet 707b2c1a462e463b94f1dd8092b8a795
I20260811 00:26:57.428740 11307 tablet_service.cc:1514] Processing DeleteTablet for tablet 707b2c1a462e463b94f1dd8092b8a795 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:57 UTC) from {username='oryx', principal='oryx/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:53720
I20260811 00:26:57.429169 11622 tablet_service.cc:1514] Processing DeleteTablet for tablet 707b2c1a462e463b94f1dd8092b8a795 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:57 UTC) from {username='oryx', principal='oryx/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:48550
I20260811 00:26:57.429229 11463 tablet_service.cc:1514] Processing DeleteTablet for tablet 707b2c1a462e463b94f1dd8092b8a795 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:57 UTC) from {username='oryx', principal='oryx/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:46684
I20260811 00:26:57.435940 11809 tablet_replica.cc:330] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337: stopping tablet replica
I20260811 00:26:57.437156 11809 raft_consensus.cc:2229] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:26:57.438380 11809 raft_consensus.cc:2258] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:57.441102 11810 tablet_replica.cc:330] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87: stopping tablet replica
I20260811 00:26:57.441422 11810 raft_consensus.cc:2229] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:57.441810 11810 raft_consensus.cc:2258] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:57.443216 11809 ts_tablet_manager.cc:1905] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:57.447147 11810 ts_tablet_manager.cc:1905] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:57.455430 11808 tablet_replica.cc:330] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79: stopping tablet replica
I20260811 00:26:57.455770 11808 raft_consensus.cc:2229] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:57.456226 11808 raft_consensus.cc:2258] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:57.458710 11809 ts_tablet_manager.cc:1918] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:26:57.459029 11809 log.cc:1196] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/707b2c1a462e463b94f1dd8092b8a795
I20260811 00:26:57.459877 11809 ts_tablet_manager.cc:1939] T 707b2c1a462e463b94f1dd8092b8a795 P 654bed556a1f4ad995ce2ecdc5e89337: Deleting consensus metadata
I20260811 00:26:57.459896 11808 ts_tablet_manager.cc:1905] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:57.467442 11810 ts_tablet_manager.cc:1918] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:26:57.467813 11810 log.cc:1196] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/707b2c1a462e463b94f1dd8092b8a795
I20260811 00:26:57.468603 11810 ts_tablet_manager.cc:1939] T 707b2c1a462e463b94f1dd8092b8a795 P 97a00c1cd50a41f2af3d16254a462e87: Deleting consensus metadata
I20260811 00:26:57.474888 11136 catalog_manager.cc:4841] TS 97a00c1cd50a41f2af3d16254a462e87 (127.2.120.2:42735): tablet 707b2c1a462e463b94f1dd8092b8a795 (table test-table [id=6794d766b6cc4eed86b9bd37faec8bda]) successfully deleted
I20260811 00:26:57.475620 11138 catalog_manager.cc:4841] TS 654bed556a1f4ad995ce2ecdc5e89337 (127.2.120.3:41297): tablet 707b2c1a462e463b94f1dd8092b8a795 (table test-table [id=6794d766b6cc4eed86b9bd37faec8bda]) successfully deleted
I20260811 00:26:57.475618 11808 ts_tablet_manager.cc:1918] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:26:57.476141 11808 log.cc:1196] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/707b2c1a462e463b94f1dd8092b8a795
I20260811 00:26:57.477056 11808 ts_tablet_manager.cc:1939] T 707b2c1a462e463b94f1dd8092b8a795 P 8869d022846f4355a865a32c91f61c79: Deleting consensus metadata
I20260811 00:26:57.479617 11138 catalog_manager.cc:4841] TS 8869d022846f4355a865a32c91f61c79 (127.2.120.1:33821): tablet 707b2c1a462e463b94f1dd8092b8a795 (table test-table [id=6794d766b6cc4eed86b9bd37faec8bda]) successfully deleted
Aug 11 00:26:57 dist-test-slave-fd7l krb5kdc[11105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 00:26:57 dist-test-slave-fd7l krb5kdc[11105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:26:57 dist-test-slave-fd7l krb5kdc[11105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 00:26:57.551575 11822 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:26:57.524678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57284 (local address 127.2.120.62:39689)
0811 00:26:57.539170 (+ 14492us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:57.539175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:57.539196 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:26:57.539256 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:57.539260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:57.539327 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:57.539489 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:57.541984 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.542828 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.544371 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.544976 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.545138 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:57.551259 (+ 6121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57284: BlockingRecv error: recv got EOF from 127.0.0.1:57284 (error 108)
Metrics: {"server-negotiator.queue_time_us":14364,"thread_start_us":14228,"threads_started":1}
Aug 11 00:26:57 dist-test-slave-fd7l krb5kdc[11105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM
I20260811 00:26:57.572624 11822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:57.555483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57286 (local address 127.2.120.62:39689)
0811 00:26:57.555696 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:57.555702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:57.555723 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:26:57.555896 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:57.555909 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:57.555984 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:57.556099 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:57.557581 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.558364 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.560254 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.560791 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.560954 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:57.565459 (+ 4505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:57.565468 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:57.565474 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:57.565526 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:57.569335 (+ 3809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:57.570238 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:57.570244 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:57.570247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:57.570436 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:57.570964 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:57.570970 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:57.570973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:57.571290 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:57.571602 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:57.571930 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 00:26:57.572200 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 00:26:57.576848 11153 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:26:57.577598 11153 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:26:57.586179 11307 tablet_service.cc:1467] Processing CreateTablet for tablet 31e730e597d04bad966ca27117121325 (DEFAULT_TABLE table=test-table [id=a933ad534b794e979ee171ae12063db3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:57.586462 11622 tablet_service.cc:1467] Processing CreateTablet for tablet 31e730e597d04bad966ca27117121325 (DEFAULT_TABLE table=test-table [id=a933ad534b794e979ee171ae12063db3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:57.586737 11307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31e730e597d04bad966ca27117121325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:57.586984 11622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31e730e597d04bad966ca27117121325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:57.591097 11463 tablet_service.cc:1467] Processing CreateTablet for tablet 31e730e597d04bad966ca27117121325 (DEFAULT_TABLE table=test-table [id=a933ad534b794e979ee171ae12063db3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:57.591576 11463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31e730e597d04bad966ca27117121325. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:57.602409 11828 tablet_bootstrap.cc:492] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337: Bootstrap starting.
I20260811 00:26:57.605523 11828 tablet_bootstrap.cc:654] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:57.607409 11830 tablet_bootstrap.cc:492] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87: Bootstrap starting.
I20260811 00:26:57.607399 11829 tablet_bootstrap.cc:492] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79: Bootstrap starting.
I20260811 00:26:57.609786 11830 tablet_bootstrap.cc:654] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:57.610201 11829 tablet_bootstrap.cc:654] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:57.612421 11828 tablet_bootstrap.cc:492] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337: No bootstrap required, opened a new log
I20260811 00:26:57.612718 11828 ts_tablet_manager.cc:1397] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260811 00:26:57.613266 11830 tablet_bootstrap.cc:492] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87: No bootstrap required, opened a new log
I20260811 00:26:57.613488 11830 ts_tablet_manager.cc:1397] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 00:26:57.614688 11830 raft_consensus.cc:348] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.615065 11830 raft_consensus.cc:374] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:57.615203 11830 raft_consensus.cc:729] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97a00c1cd50a41f2af3d16254a462e87, State: Initialized, Role: FOLLOWER
I20260811 00:26:57.615602 11828 raft_consensus.cc:348] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.615607 11830 consensus_queue.cc:260] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.616236 11828 raft_consensus.cc:374] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:57.616401 11828 raft_consensus.cc:729] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 654bed556a1f4ad995ce2ecdc5e89337, State: Initialized, Role: FOLLOWER
I20260811 00:26:57.616791 11828 consensus_queue.cc:260] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.618608 11828 ts_tablet_manager.cc:1428] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260811 00:26:57.621413 11830 ts_tablet_manager.cc:1428] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87: Time spent starting tablet: real 0.008s user 0.000s sys 0.003s
I20260811 00:26:57.623438 11829 tablet_bootstrap.cc:492] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79: No bootstrap required, opened a new log
I20260811 00:26:57.623643 11829 ts_tablet_manager.cc:1397] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.007s
I20260811 00:26:57.624598 11829 raft_consensus.cc:348] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.624925 11829 raft_consensus.cc:374] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:57.625026 11829 raft_consensus.cc:729] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8869d022846f4355a865a32c91f61c79, State: Initialized, Role: FOLLOWER
I20260811 00:26:57.625319 11829 consensus_queue.cc:260] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.626150 11829 ts_tablet_manager.cc:1428] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 00:26:57.726501 11739 raft_consensus.cc:482] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:26:57.726799 11739 raft_consensus.cc:504] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.727597 11739 leader_election.cc:288] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8869d022846f4355a865a32c91f61c79 (127.2.120.1:33821), 97a00c1cd50a41f2af3d16254a462e87 (127.2.120.2:42735)
I20260811 00:26:57.728199 11334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31e730e597d04bad966ca27117121325" candidate_uuid: "654bed556a1f4ad995ce2ecdc5e89337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8869d022846f4355a865a32c91f61c79" is_pre_election: true
I20260811 00:26:57.728212 11485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31e730e597d04bad966ca27117121325" candidate_uuid: "654bed556a1f4ad995ce2ecdc5e89337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97a00c1cd50a41f2af3d16254a462e87" is_pre_election: true
I20260811 00:26:57.728502 11485 raft_consensus.cc:2454] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 654bed556a1f4ad995ce2ecdc5e89337 in term 0.
I20260811 00:26:57.728492 11334 raft_consensus.cc:2454] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 654bed556a1f4ad995ce2ecdc5e89337 in term 0.
I20260811 00:26:57.729053 11566 leader_election.cc:302] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 654bed556a1f4ad995ce2ecdc5e89337, 97a00c1cd50a41f2af3d16254a462e87; no voters:
I20260811 00:26:57.729485 11739 raft_consensus.cc:2781] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:26:57.729619 11739 raft_consensus.cc:482] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:26:57.729717 11739 raft_consensus.cc:3037] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:57.732793 11739 raft_consensus.cc:504] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.733646 11739 leader_election.cc:288] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [CANDIDATE]: Term 1 election: Requested vote from peers 8869d022846f4355a865a32c91f61c79 (127.2.120.1:33821), 97a00c1cd50a41f2af3d16254a462e87 (127.2.120.2:42735)
I20260811 00:26:57.734361 11485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31e730e597d04bad966ca27117121325" candidate_uuid: "654bed556a1f4ad995ce2ecdc5e89337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97a00c1cd50a41f2af3d16254a462e87"
I20260811 00:26:57.734404 11334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31e730e597d04bad966ca27117121325" candidate_uuid: "654bed556a1f4ad995ce2ecdc5e89337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8869d022846f4355a865a32c91f61c79"
I20260811 00:26:57.734651 11334 raft_consensus.cc:3037] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:57.734656 11485 raft_consensus.cc:3037] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:57.737130 11485 raft_consensus.cc:2454] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 654bed556a1f4ad995ce2ecdc5e89337 in term 1.
I20260811 00:26:57.737725 11334 raft_consensus.cc:2454] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 654bed556a1f4ad995ce2ecdc5e89337 in term 1.
I20260811 00:26:57.737809 11566 leader_election.cc:302] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 654bed556a1f4ad995ce2ecdc5e89337, 97a00c1cd50a41f2af3d16254a462e87; no voters:
I20260811 00:26:57.738248 11739 raft_consensus.cc:2781] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:57.738507 11739 raft_consensus.cc:686] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 LEADER]: Becoming Leader. State: Replica: 654bed556a1f4ad995ce2ecdc5e89337, State: Running, Role: LEADER
I20260811 00:26:57.738832 11739 consensus_queue.cc:237] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.741941 11153 catalog_manager.cc:5495] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 reported cstate change: term changed from 0 to 1, leader changed from <none> to 654bed556a1f4ad995ce2ecdc5e89337 (127.2.120.3). New cstate: current_term: 1 leader_uuid: "654bed556a1f4ad995ce2ecdc5e89337" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } health_report { overall_health: HEALTHY } } }
I20260811 00:26:57.769920 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:57.764676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48562 (local address 127.2.120.3:41297)
0811 00:26:57.764866 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:57.764874 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:57.764948 (+ 74us) server_negotiation.cc:407] Connection header received
0811 00:26:57.765229 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:57.765240 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:57.765314 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:57.765419 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:26:57.765907 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.766679 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.768052 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:57.768605 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:57.768734 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:57.769215 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:26:57.769338 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:26:57.769411 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:57.769599 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 00:26:57.769676 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:26:57.772756 11485 raft_consensus.cc:1261] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87 [term 1 FOLLOWER]: Refusing update from remote peer 654bed556a1f4ad995ce2ecdc5e89337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:57.772770 11334 raft_consensus.cc:1261] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79 [term 1 FOLLOWER]: Refusing update from remote peer 654bed556a1f4ad995ce2ecdc5e89337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:57.773520 11739 consensus_queue.cc:1035] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:57.773916 11799 consensus_queue.cc:1035] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:57.780633 11787 mvcc.cc:205] Tried to move back new op lower bound from 7317127240791007232 to 7317127240661245952. Current Snapshot: MvccSnapshot[applied={T|T < 7317127240791007232}]
I20260811 00:26:57.786968 11153 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:57.787271 11153 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:57.790174 11153 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a: Sending DeleteTablet for 3 replicas of tablet 31e730e597d04bad966ca27117121325
I20260811 00:26:57.791144 11307 tablet_service.cc:1514] Processing DeleteTablet for tablet 31e730e597d04bad966ca27117121325 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:57 UTC) from {username='oryx', principal='oryx/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:53720
I20260811 00:26:57.791188 11463 tablet_service.cc:1514] Processing DeleteTablet for tablet 31e730e597d04bad966ca27117121325 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:57 UTC) from {username='oryx', principal='oryx/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:46684
I20260811 00:26:57.791457 11808 tablet_replica.cc:330] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79: stopping tablet replica
I20260811 00:26:57.791477 11810 tablet_replica.cc:330] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87: stopping tablet replica
I20260811 00:26:57.791544 11622 tablet_service.cc:1514] Processing DeleteTablet for tablet 31e730e597d04bad966ca27117121325 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:57 UTC) from {username='oryx', principal='oryx/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:48550
I20260811 00:26:57.791698 11808 raft_consensus.cc:2229] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:57.791698 11810 raft_consensus.cc:2229] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:57.791846 11809 tablet_replica.cc:330] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337: stopping tablet replica
I20260811 00:26:57.791939 11808 raft_consensus.cc:2258] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:57.791939 11810 raft_consensus.cc:2258] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:57.792061 11809 raft_consensus.cc:2229] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:26:57.792411 11809 raft_consensus.cc:2258] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:57.793082 11810 ts_tablet_manager.cc:1905] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:57.793399 11808 ts_tablet_manager.cc:1905] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:57.793601 11809 ts_tablet_manager.cc:1905] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:57.798699 11153 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 00:26:57.799079 11808 ts_tablet_manager.cc:1918] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:26:57.799252 11808 log.cc:1196] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/31e730e597d04bad966ca27117121325
I20260811 00:26:57.799718 11808 ts_tablet_manager.cc:1939] T 31e730e597d04bad966ca27117121325 P 8869d022846f4355a865a32c91f61c79: Deleting consensus metadata
W20260811 00:26:57.799924 11153 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:26:57.800469 11809 ts_tablet_manager.cc:1918] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:26:57.800468 11810 ts_tablet_manager.cc:1918] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:26:57.800675 11810 log.cc:1196] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/31e730e597d04bad966ca27117121325
I20260811 00:26:57.800675 11809 log.cc:1196] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/31e730e597d04bad966ca27117121325
I20260811 00:26:57.801011 11138 catalog_manager.cc:4841] TS 8869d022846f4355a865a32c91f61c79 (127.2.120.1:33821): tablet 31e730e597d04bad966ca27117121325 (table test-table [id=a933ad534b794e979ee171ae12063db3]) successfully deleted
I20260811 00:26:57.801162 11810 ts_tablet_manager.cc:1939] T 31e730e597d04bad966ca27117121325 P 97a00c1cd50a41f2af3d16254a462e87: Deleting consensus metadata
I20260811 00:26:57.801357 11809 ts_tablet_manager.cc:1939] T 31e730e597d04bad966ca27117121325 P 654bed556a1f4ad995ce2ecdc5e89337: Deleting consensus metadata
I20260811 00:26:57.802506 11136 catalog_manager.cc:4841] TS 97a00c1cd50a41f2af3d16254a462e87 (127.2.120.2:42735): tablet 31e730e597d04bad966ca27117121325 (table test-table [id=a933ad534b794e979ee171ae12063db3]) successfully deleted
I20260811 00:26:57.802702 11138 catalog_manager.cc:4841] TS 654bed556a1f4ad995ce2ecdc5e89337 (127.2.120.3:41297): tablet 31e730e597d04bad966ca27117121325 (table test-table [id=a933ad534b794e979ee171ae12063db3]) successfully deleted
I20260811 00:26:57.809041 11463 tablet_service.cc:1467] Processing CreateTablet for tablet bd8ce1af17c94a6c818856196e78e567 (DEFAULT_TABLE table=test-table [id=03a1635a3ec34996850c1a0df1c1e05d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:57.809032 11307 tablet_service.cc:1467] Processing CreateTablet for tablet bd8ce1af17c94a6c818856196e78e567 (DEFAULT_TABLE table=test-table [id=03a1635a3ec34996850c1a0df1c1e05d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:57.809619 11463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd8ce1af17c94a6c818856196e78e567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:57.809619 11307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd8ce1af17c94a6c818856196e78e567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:57.812577 11622 tablet_service.cc:1467] Processing CreateTablet for tablet bd8ce1af17c94a6c818856196e78e567 (DEFAULT_TABLE table=test-table [id=03a1635a3ec34996850c1a0df1c1e05d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:26:57.813144 11622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd8ce1af17c94a6c818856196e78e567. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:26:57.817523 11830 tablet_bootstrap.cc:492] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87: Bootstrap starting.
I20260811 00:26:57.817605 11829 tablet_bootstrap.cc:492] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79: Bootstrap starting.
I20260811 00:26:57.819087 11828 tablet_bootstrap.cc:492] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337: Bootstrap starting.
I20260811 00:26:57.819530 11830 tablet_bootstrap.cc:654] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:57.820138 11829 tablet_bootstrap.cc:654] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:57.822062 11828 tablet_bootstrap.cc:654] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337: Neither blocks nor log segments found. Creating new log.
I20260811 00:26:57.822312 11830 tablet_bootstrap.cc:492] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87: No bootstrap required, opened a new log
I20260811 00:26:57.822527 11830 ts_tablet_manager.cc:1397] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260811 00:26:57.823494 11830 raft_consensus.cc:348] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.823817 11830 raft_consensus.cc:374] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:57.823910 11830 raft_consensus.cc:729] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97a00c1cd50a41f2af3d16254a462e87, State: Initialized, Role: FOLLOWER
I20260811 00:26:57.824229 11830 consensus_queue.cc:260] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.825058 11830 ts_tablet_manager.cc:1428] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 00:26:57.826683 11828 tablet_bootstrap.cc:492] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337: No bootstrap required, opened a new log
I20260811 00:26:57.826910 11828 ts_tablet_manager.cc:1397] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260811 00:26:57.827757 11828 raft_consensus.cc:348] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.828054 11828 raft_consensus.cc:374] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:57.828161 11828 raft_consensus.cc:729] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 654bed556a1f4ad995ce2ecdc5e89337, State: Initialized, Role: FOLLOWER
I20260811 00:26:57.828454 11829 tablet_bootstrap.cc:492] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79: No bootstrap required, opened a new log
I20260811 00:26:57.828423 11828 consensus_queue.cc:260] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.828688 11829 ts_tablet_manager.cc:1397] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260811 00:26:57.829097 11828 ts_tablet_manager.cc:1428] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 00:26:57.830284 11829 raft_consensus.cc:348] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.830581 11829 raft_consensus.cc:374] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:26:57.830682 11829 raft_consensus.cc:729] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8869d022846f4355a865a32c91f61c79, State: Initialized, Role: FOLLOWER
I20260811 00:26:57.831189 11829 consensus_queue.cc:260] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.831805 11829 ts_tablet_manager.cc:1428] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 00:26:57.959246 11752 raft_consensus.cc:482] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:26:57.959551 11752 raft_consensus.cc:504] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.960299 11752 leader_election.cc:288] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8869d022846f4355a865a32c91f61c79 (127.2.120.1:33821), 97a00c1cd50a41f2af3d16254a462e87 (127.2.120.2:42735)
I20260811 00:26:57.961301 11485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd8ce1af17c94a6c818856196e78e567" candidate_uuid: "654bed556a1f4ad995ce2ecdc5e89337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97a00c1cd50a41f2af3d16254a462e87" is_pre_election: true
I20260811 00:26:57.961416 11334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd8ce1af17c94a6c818856196e78e567" candidate_uuid: "654bed556a1f4ad995ce2ecdc5e89337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8869d022846f4355a865a32c91f61c79" is_pre_election: true
I20260811 00:26:57.961601 11485 raft_consensus.cc:2454] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 654bed556a1f4ad995ce2ecdc5e89337 in term 0.
I20260811 00:26:57.961772 11334 raft_consensus.cc:2454] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 654bed556a1f4ad995ce2ecdc5e89337 in term 0.
I20260811 00:26:57.962270 11566 leader_election.cc:302] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 654bed556a1f4ad995ce2ecdc5e89337, 97a00c1cd50a41f2af3d16254a462e87; no voters:
I20260811 00:26:57.963022 11752 raft_consensus.cc:2781] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:26:57.963151 11752 raft_consensus.cc:482] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:26:57.963279 11752 raft_consensus.cc:3037] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:57.965794 11752 raft_consensus.cc:504] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.966481 11752 leader_election.cc:288] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [CANDIDATE]: Term 1 election: Requested vote from peers 8869d022846f4355a865a32c91f61c79 (127.2.120.1:33821), 97a00c1cd50a41f2af3d16254a462e87 (127.2.120.2:42735)
I20260811 00:26:57.967136 11485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd8ce1af17c94a6c818856196e78e567" candidate_uuid: "654bed556a1f4ad995ce2ecdc5e89337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97a00c1cd50a41f2af3d16254a462e87"
I20260811 00:26:57.967176 11334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd8ce1af17c94a6c818856196e78e567" candidate_uuid: "654bed556a1f4ad995ce2ecdc5e89337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8869d022846f4355a865a32c91f61c79"
I20260811 00:26:57.967393 11485 raft_consensus.cc:3037] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:57.967439 11334 raft_consensus.cc:3037] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:26:57.969641 11485 raft_consensus.cc:2454] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 654bed556a1f4ad995ce2ecdc5e89337 in term 1.
I20260811 00:26:57.970086 11334 raft_consensus.cc:2454] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 654bed556a1f4ad995ce2ecdc5e89337 in term 1.
I20260811 00:26:57.970286 11566 leader_election.cc:302] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 654bed556a1f4ad995ce2ecdc5e89337, 97a00c1cd50a41f2af3d16254a462e87; no voters:
I20260811 00:26:57.970695 11752 raft_consensus.cc:2781] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:26:57.970875 11752 raft_consensus.cc:686] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 LEADER]: Becoming Leader. State: Replica: 654bed556a1f4ad995ce2ecdc5e89337, State: Running, Role: LEADER
I20260811 00:26:57.971169 11752 consensus_queue.cc:237] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } }
I20260811 00:26:57.974299 11152 catalog_manager.cc:5495] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 reported cstate change: term changed from 0 to 1, leader changed from <none> to 654bed556a1f4ad995ce2ecdc5e89337 (127.2.120.3). New cstate: current_term: 1 leader_uuid: "654bed556a1f4ad995ce2ecdc5e89337" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "654bed556a1f4ad995ce2ecdc5e89337" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41297 } health_report { overall_health: HEALTHY } } }
I20260811 00:26:58.036186 11485 raft_consensus.cc:1261] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87 [term 1 FOLLOWER]: Refusing update from remote peer 654bed556a1f4ad995ce2ecdc5e89337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:58.036993 11334 raft_consensus.cc:1261] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79 [term 1 FOLLOWER]: Refusing update from remote peer 654bed556a1f4ad995ce2ecdc5e89337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:26:58.037060 11752 consensus_queue.cc:1035] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97a00c1cd50a41f2af3d16254a462e87" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:58.037631 11739 consensus_queue.cc:1035] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8869d022846f4355a865a32c91f61c79" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:26:58.103713 11153 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:58.104017 11153 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:26:58.107549 11153 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81df64d70ee74d6dbfb37064593c0b9a: Sending DeleteTablet for 3 replicas of tablet bd8ce1af17c94a6c818856196e78e567
I20260811 00:26:58.108635 11307 tablet_service.cc:1514] Processing DeleteTablet for tablet bd8ce1af17c94a6c818856196e78e567 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:58 UTC) from {username='oryx', principal='oryx/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:53720
I20260811 00:26:58.108661 11622 tablet_service.cc:1514] Processing DeleteTablet for tablet bd8ce1af17c94a6c818856196e78e567 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:58 UTC) from {username='oryx', principal='oryx/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:48550
I20260811 00:26:58.108847 11463 tablet_service.cc:1514] Processing DeleteTablet for tablet bd8ce1af17c94a6c818856196e78e567 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:26:58 UTC) from {username='oryx', principal='oryx/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:46684
I20260811 00:26:58.108991 11808 tablet_replica.cc:330] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79: stopping tablet replica
I20260811 00:26:58.109092 11809 tablet_replica.cc:330] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337: stopping tablet replica
I20260811 00:26:58.109205 11808 raft_consensus.cc:2229] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:58.109279 11809 raft_consensus.cc:2229] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:26:58.109318 11810 tablet_replica.cc:330] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87: stopping tablet replica
I20260811 00:26:58.109449 11808 raft_consensus.cc:2258] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:58.109510 11810 raft_consensus.cc:2229] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:26:58.109655 11809 raft_consensus.cc:2258] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:58.109764 11810 raft_consensus.cc:2258] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:26:58.110749 11810 ts_tablet_manager.cc:1905] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:58.110786 11809 ts_tablet_manager.cc:1905] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:58.110785 11808 ts_tablet_manager.cc:1905] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:26:58.116256 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 11234
I20260811 00:26:58.117604 11810 ts_tablet_manager.cc:1918] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:26:58.117775 11810 log.cc:1196] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/bd8ce1af17c94a6c818856196e78e567
I20260811 00:26:58.118288 11810 ts_tablet_manager.cc:1939] T bd8ce1af17c94a6c818856196e78e567 P 97a00c1cd50a41f2af3d16254a462e87: Deleting consensus metadata
I20260811 00:26:58.119136 11808 ts_tablet_manager.cc:1918] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:26:58.119323 11808 log.cc:1196] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/bd8ce1af17c94a6c818856196e78e567
I20260811 00:26:58.119575 11136 catalog_manager.cc:4841] TS 97a00c1cd50a41f2af3d16254a462e87 (127.2.120.2:42735): tablet bd8ce1af17c94a6c818856196e78e567 (table test-table [id=03a1635a3ec34996850c1a0df1c1e05d]) successfully deleted
I20260811 00:26:58.120788 11808 ts_tablet_manager.cc:1939] T bd8ce1af17c94a6c818856196e78e567 P 8869d022846f4355a865a32c91f61c79: Deleting consensus metadata
I20260811 00:26:58.121958 11138 catalog_manager.cc:4841] TS 8869d022846f4355a865a32c91f61c79 (127.2.120.1:33821): tablet bd8ce1af17c94a6c818856196e78e567 (table test-table [id=03a1635a3ec34996850c1a0df1c1e05d]) successfully deleted
I20260811 00:26:58.122318 11809 ts_tablet_manager.cc:1918] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:26:58.122493 11809 log.cc:1196] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/bd8ce1af17c94a6c818856196e78e567
I20260811 00:26:58.123001 11809 ts_tablet_manager.cc:1939] T bd8ce1af17c94a6c818856196e78e567 P 654bed556a1f4ad995ce2ecdc5e89337: Deleting consensus metadata
I20260811 00:26:58.124109 11138 catalog_manager.cc:4841] TS 654bed556a1f4ad995ce2ecdc5e89337 (127.2.120.3:41297): tablet bd8ce1af17c94a6c818856196e78e567 (table test-table [id=03a1635a3ec34996850c1a0df1c1e05d]) successfully deleted
Aug 11 00:26:58 dist-test-slave-fd7l krb5kdc[11105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.120.1@KRBTEST.COM
I20260811 00:26:58.139463 11848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:58.117733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53726 (local address 127.2.120.1:33821)
0811 00:26:58.119788 (+ 2055us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:58.119795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:58.120315 (+ 520us) server_negotiation.cc:407] Connection header received
0811 00:26:58.120577 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:58.120586 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:58.120684 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:58.120830 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:58.122929 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:58.123864 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:58.125602 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:58.126093 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:58.126259 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:58.131346 (+ 5087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:58.131368 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:58.131388 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:58.131445 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:58.135122 (+ 3677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:58.136441 (+ 1319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:58.136473 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:58.136476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:58.136812 (+ 336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:58.137558 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:58.137563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:58.137566 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:58.138113 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:58.138421 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:58.138852 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 00:26:58.139126 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1898,"thread_start_us":1754,"threads_started":1}
I20260811 00:26:58.140368 11378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:58.246702 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 11234
I20260811 00:26:58.273638 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 11390
Aug 11 00:26:58 dist-test-slave-fd7l krb5kdc[11105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.120.2@KRBTEST.COM
I20260811 00:26:58.296221 11852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:58.274607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46710 (local address 127.2.120.2:42735)
0811 00:26:58.275201 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:58.275206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:58.275226 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:26:58.275289 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:58.275293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:58.275370 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:58.275478 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:58.277631 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:58.278982 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:58.280837 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:58.281491 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:58.281674 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:58.287431 (+ 5757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:58.287460 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:58.287488 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:58.287555 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:58.292699 (+ 5144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:58.293712 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:58.293722 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:58.293725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:58.293889 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:58.294363 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:58.294367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:58.294369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:58.294805 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:58.295165 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:58.295516 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 00:26:58.295821 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":303,"threads_started":1}
I20260811 00:26:58.297083 11536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:58.432792 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 11390
I20260811 00:26:58.461843 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 11548
Aug 11 00:26:58 dist-test-slave-fd7l krb5kdc[11105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.120.3@KRBTEST.COM
I20260811 00:26:58.483217 11854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:26:58.463081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48574 (local address 127.2.120.3:41297)
0811 00:26:58.463782 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0811 00:26:58.463788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:26:58.463811 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:26:58.463881 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:26:58.463885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:26:58.463966 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:26:58.464087 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:26:58.466105 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:58.466862 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:58.468360 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:26:58.468943 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:26:58.469071 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:26:58.475022 (+ 5951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:26:58.475043 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:26:58.475064 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:26:58.475115 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:26:58.479874 (+ 4759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:58.480982 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:58.480992 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:58.480995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:58.481155 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:26:58.481597 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:26:58.481601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:26:58.481604 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:26:58.482033 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:26:58.482302 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 00:26:58.482716 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 00:26:58.482937 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":381,"threads_started":1}
I20260811 00:26:58.484316 11693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:58.618233 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 11548
I20260811 00:26:58.649046 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 11120
I20260811 00:26:58.650439 11199 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:26:58.776401 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 11120
2026-08-11T00:26:58Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7176 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:26:58 dist-test-slave-fd7l krb5kdc[11859](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:26:58 dist-test-slave-fd7l krb5kdc[11859](info): set up 2 sockets
Aug 11 00:26:58 dist-test-slave-fd7l krb5kdc[11859](info): commencing operation
krb5kdc: starting...
W20260811 00:27:00.038923 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.007s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:27:00 dist-test-slave-fd7l krb5kdc[11859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:27:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:27:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:27:00.360924 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:33039
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:33039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:27:00.743961 11873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:00.744299 11873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:00.744472 11873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:00.752480 11873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:27:00.752715 11873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:00.752801 11873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:00.752890 11873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:27:00.752981 11873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:27:00.764029 11873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:33039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:33039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:00.765775 11873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:00.767373 11873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:00.781579 11883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:00.781723 11881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:00.781530 11879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:00.783480 11873 server_base.cc:1034] running on GCE node
I20260811 00:27:00.784719 11873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:00.787030 11873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:00.788383 11873 hybrid_clock.cc:648] HybridClock initialized: now 1786408020788285 us; error 66 us; skew 500 ppm
I20260811 00:27:00.789062 11873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T00:27:00Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2097 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:27:00 dist-test-slave-fd7l krb5kdc[11889](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:27:00 dist-test-slave-fd7l krb5kdc[11889](info): set up 2 sockets
Aug 11 00:27:00 dist-test-slave-fd7l krb5kdc[11889](info): commencing operation
krb5kdc: starting...
W20260811 00:27:02.120890 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:27:02 dist-test-slave-fd7l krb5kdc[11889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:27:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:27:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:27:02.443210 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:37663
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:37203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:37663
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:27:02.823093 11904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:02.823509 11904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:02.823690 11904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:02.831591 11904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:27:02.831764 11904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:02.831887 11904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:02.832003 11904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:27:02.832086 11904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:27:02.843045 11904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:37663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:37663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:02.844713 11904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:02.846266 11904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:02.861212 11910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:02.861171 11914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:02.861261 11911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:02.862244 11904 server_base.cc:1034] running on GCE node
I20260811 00:27:02.863260 11904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:02.865089 11904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:02.866406 11904 hybrid_clock.cc:648] HybridClock initialized: now 1786408022866349 us; error 48 us; skew 500 ppm
I20260811 00:27:02.866887 11904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:27:02 dist-test-slave-fd7l krb5kdc[11889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408022, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:27:02.876768 11904 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:27:02.878798 11904 webserver.cc:460] Webserver started at http://127.2.120.62:46191/ using document root <none> and password file <none>
I20260811 00:27:02.879679 11904 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:02.879863 11904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:02.880241 11904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:27:02.884138 11904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "1d98caae9dd64e0aa55876ee62e351bd"
format_stamp: "Formatted at 2026-08-11 00:27:02 on dist-test-slave-fd7l"
server_key: "c5f11b90c8b707c2088ee052e088c487"
server_key_iv: "7cdc9bd5ffa66b856ae1d16bbe3d30ae"
server_key_version: "encryptionkey@0"
I20260811 00:27:02.885281 11904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "1d98caae9dd64e0aa55876ee62e351bd"
format_stamp: "Formatted at 2026-08-11 00:27:02 on dist-test-slave-fd7l"
server_key: "c5f11b90c8b707c2088ee052e088c487"
server_key_iv: "7cdc9bd5ffa66b856ae1d16bbe3d30ae"
server_key_version: "encryptionkey@0"
I20260811 00:27:02.892099 11904 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 00:27:02.897540 11922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:02.899879 11904 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:27:02.900404 11904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "1d98caae9dd64e0aa55876ee62e351bd"
format_stamp: "Formatted at 2026-08-11 00:27:02 on dist-test-slave-fd7l"
server_key: "c5f11b90c8b707c2088ee052e088c487"
server_key_iv: "7cdc9bd5ffa66b856ae1d16bbe3d30ae"
server_key_version: "encryptionkey@0"
I20260811 00:27:02.900914 11904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:02.922667 11904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:02.928047 11904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:02.928431 11904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:02.952203 11904 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:37663
I20260811 00:27:02.952234 11977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:37663 every 8 connection(s)
I20260811 00:27:02.953953 11904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:27:02.958283 11978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:27:02.964473 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 11904
I20260811 00:27:02.964850 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:27:02.965552 2528 external_mini_cluster.cc:1428] Setting key efdb31bae29d2de822a4ca78caa2eead
I20260811 00:27:02.971411 11978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd: Bootstrap starting.
I20260811 00:27:02.975952 11978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd: Neither blocks nor log segments found. Creating new log.
I20260811 00:27:02.977666 11978 log.cc:824] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd: Log is configured to *not* fsync() on all Append() calls
I20260811 00:27:02.982264 11978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd: No bootstrap required, opened a new log
Aug 11 00:27:02 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:27:02.993176 11978 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d98caae9dd64e0aa55876ee62e351bd" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37663 } }
I20260811 00:27:02.993736 11978 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:27:02.993853 11978 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d98caae9dd64e0aa55876ee62e351bd, State: Initialized, Role: FOLLOWER
I20260811 00:27:02.994567 11978 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d98caae9dd64e0aa55876ee62e351bd" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37663 } }
I20260811 00:27:02.994839 11978 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:27:02.995040 11978 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:27:02.995244 11978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:27:02.995834 11981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:02.968044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56850 (local address 127.2.120.62:37663)
0811 00:27:02.969070 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:02.969090 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:02.969140 (+ 50us) server_negotiation.cc:407] Connection header received
0811 00:27:02.971059 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:02.971275 (+ 216us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:02.971876 (+ 601us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:02.972288 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:02.974532 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:02.976194 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:02.978278 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:02.979680 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:02.979864 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:02.985942 (+ 6078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:02.985953 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:02.985976 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:02.986040 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:02.989979 (+ 3939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:02.991074 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:02.991083 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:02.991091 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:02.991298 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:02.991979 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:02.991983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:02.991997 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:02.992893 (+ 896us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:02.993136 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:02.993810 (+ 674us) server_negotiation.cc:299] Negotiation successful
0811 00:27:02.994333 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":501,"threads_started":1}
I20260811 00:27:02.998588 11978 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d98caae9dd64e0aa55876ee62e351bd" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37663 } }
I20260811 00:27:02.999347 11978 leader_election.cc:302] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d98caae9dd64e0aa55876ee62e351bd; no voters:
I20260811 00:27:03.000432 11978 leader_election.cc:288] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:27:03.000793 11983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:27:03.001941 11983 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [term 1 LEADER]: Becoming Leader. State: Replica: 1d98caae9dd64e0aa55876ee62e351bd, State: Running, Role: LEADER
I20260811 00:27:03.002655 11983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d98caae9dd64e0aa55876ee62e351bd" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37663 } }
I20260811 00:27:03.003430 11978 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:27:03.006083 11985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d98caae9dd64e0aa55876ee62e351bd. Latest consensus state: current_term: 1 leader_uuid: "1d98caae9dd64e0aa55876ee62e351bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d98caae9dd64e0aa55876ee62e351bd" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37663 } } }
I20260811 00:27:03.006649 11985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [sys.catalog]: This master's current role is: LEADER
I20260811 00:27:03.007284 11984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d98caae9dd64e0aa55876ee62e351bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d98caae9dd64e0aa55876ee62e351bd" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37663 } } }
I20260811 00:27:03.007623 11984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [sys.catalog]: This master's current role is: LEADER
I20260811 00:27:03.014923 11990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:27:03.020746 11990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:27:03.031893 11990 catalog_manager.cc:1347] Generated new cluster ID: cb3415432fc542e1be0d71b816f870cc
I20260811 00:27:03.032068 11990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:27:03.069756 11990 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:27:03.071677 11990 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:27:03.081892 11990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd: Generated new TSK 0
I20260811 00:27:03.083132 11990 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
I20260811 00:27:03.164813 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:37663
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:37203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:27:03.571851 12004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:03.572463 12004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:03.572710 12004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:03.585484 12004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:03.585702 12004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:03.585901 12004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:27:03.597227 12004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:37663
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:03.599104 12004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:03.601104 12004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:03.617040 12010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:03.617066 12012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:03.617132 12009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:03.618445 12004 server_base.cc:1034] running on GCE node
I20260811 00:27:03.619306 12004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:03.620702 12004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:03.622010 12004 hybrid_clock.cc:648] HybridClock initialized: now 1786408023621950 us; error 79 us; skew 500 ppm
I20260811 00:27:03.622457 12004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:27:03 dist-test-slave-fd7l krb5kdc[11889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408023, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:27:03.626575 12004 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:27:03.628340 12004 webserver.cc:460] Webserver started at http://127.2.120.1:37669/ using document root <none> and password file <none>
I20260811 00:27:03.629192 12004 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:03.629362 12004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:03.629731 12004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:27:03.633253 12004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "70095e40e16249bc859761aa0dc1c0e2"
format_stamp: "Formatted at 2026-08-11 00:27:03 on dist-test-slave-fd7l"
server_key: "07973a9544d4dd075da398f2884df8b6"
server_key_iv: "d385b3d41992fafdc526d3ca52bf6d05"
server_key_version: "encryptionkey@0"
I20260811 00:27:03.634246 12004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "70095e40e16249bc859761aa0dc1c0e2"
format_stamp: "Formatted at 2026-08-11 00:27:03 on dist-test-slave-fd7l"
server_key: "07973a9544d4dd075da398f2884df8b6"
server_key_iv: "d385b3d41992fafdc526d3ca52bf6d05"
server_key_version: "encryptionkey@0"
I20260811 00:27:03.640632 12004 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 00:27:03.644877 12019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:03.646601 12004 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:27:03.646860 12004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "70095e40e16249bc859761aa0dc1c0e2"
format_stamp: "Formatted at 2026-08-11 00:27:03 on dist-test-slave-fd7l"
server_key: "07973a9544d4dd075da398f2884df8b6"
server_key_iv: "d385b3d41992fafdc526d3ca52bf6d05"
server_key_version: "encryptionkey@0"
I20260811 00:27:03.647120 12004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:03.681320 12004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:03.685932 12004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:03.686281 12004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:03.687481 12004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:27:03.689177 12004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:27:03.689289 12004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:03.689465 12004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:27:03.689543 12004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:03.729954 12004 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:40441
I20260811 00:27:03.730010 12142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:40441 every 8 connection(s)
I20260811 00:27:03.731721 12004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:27:03.740147 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 12004
I20260811 00:27:03.740422 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:27:03.741184 2528 external_mini_cluster.cc:1428] Setting key 2dbd10bf6efef72d7789b2d8a267d29c
Aug 11 00:27:03 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408023, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:27:03.757627 12146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:03.735165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:40721 (local address 127.2.120.62:37663)
0811 00:27:03.735767 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:03.735774 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:03.737162 (+ 1388us) server_negotiation.cc:407] Connection header received
0811 00:27:03.739478 (+ 2316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:03.739488 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:03.739654 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:03.739821 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:03.742146 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:03.743289 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:03.745295 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:03.745745 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:03.745923 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:03.750564 (+ 4641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:03.750570 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:03.750574 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:03.750624 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:03.753808 (+ 3184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:03.754764 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:03.754770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:03.754772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:03.754910 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:03.755439 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:03.755443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:03.755446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:03.755753 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:03.755926 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:03.756923 (+ 997us) server_negotiation.cc:299] Negotiation successful
0811 00:27:03.757203 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":290,"threads_started":1}
I20260811 00:27:03.758740 12147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:03.735878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:40721)
0811 00:27:03.736923 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:03.736990 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:03.739044 (+ 2054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:03.740102 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:03.740113 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:03.740983 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:03.741846 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:03.741872 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:03.743519 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:03.743524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:03.745086 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:03.745104 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:03.746067 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:03.746076 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:03.746370 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:03.747235 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:03.747264 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:03.750244 (+ 2980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:27:03.754017 (+ 3773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:27:03.754028 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:27:03.754049 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:27:03.754584 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:27:03.755040 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:27:03.755045 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:27:03.755049 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:27:03.755283 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:27:03.755900 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:27:03.755910 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:27:03.756401 (+ 491us) client_negotiation.cc:769] Sending connection context
0811 00:27:03.756806 (+ 405us) client_negotiation.cc:240] Negotiation successful
0811 00:27:03.757417 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":391,"threads_started":1}
I20260811 00:27:03.760705 12144 heartbeater.cc:347] Connected to a master server at 127.2.120.62:37663
I20260811 00:27:03.761166 12144 heartbeater.cc:464] Registering TS with master...
I20260811 00:27:03.762207 12144 heartbeater.cc:511] Master 127.2.120.62:37663 requested a full tablet report, sending...
I20260811 00:27:03.765201 11941 ts_manager.cc:194] Registered new tserver with Master: 70095e40e16249bc859761aa0dc1c0e2 (127.2.120.1:40441)
I20260811 00:27:03.766959 11941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:40721
WARNING: no policy specified for kudu/127.2.120.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
I20260811 00:27:03.808679 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:37663
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:37203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:27:04.182471 12150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:04.182888 12150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:04.183068 12150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:04.191061 12150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:04.191260 12150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:04.191464 12150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:27:04.202081 12150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:37663
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:04.203698 12150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:04.205259 12150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:04.219734 12155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:04.219686 12156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:04.219759 12158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:04.220553 12150 server_base.cc:1034] running on GCE node
I20260811 00:27:04.221233 12150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:04.222289 12150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:04.223534 12150 hybrid_clock.cc:648] HybridClock initialized: now 1786408024223458 us; error 68 us; skew 500 ppm
I20260811 00:27:04.223863 12150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:27:04 dist-test-slave-fd7l krb5kdc[11889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408024, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:27:04.227397 12150 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:27:04.229141 12150 webserver.cc:460] Webserver started at http://127.2.120.2:33637/ using document root <none> and password file <none>
I20260811 00:27:04.229907 12150 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:04.230022 12150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:04.230326 12150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:27:04.233661 12150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "588271aba01b4dceb9f161bb519aa169"
format_stamp: "Formatted at 2026-08-11 00:27:04 on dist-test-slave-fd7l"
server_key: "3d51fb8dc1909c88b6e23894f87645cd"
server_key_iv: "6c2d6e8b7db1c87471b89b504878e081"
server_key_version: "encryptionkey@0"
I20260811 00:27:04.234498 12150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "588271aba01b4dceb9f161bb519aa169"
format_stamp: "Formatted at 2026-08-11 00:27:04 on dist-test-slave-fd7l"
server_key: "3d51fb8dc1909c88b6e23894f87645cd"
server_key_iv: "6c2d6e8b7db1c87471b89b504878e081"
server_key_version: "encryptionkey@0"
I20260811 00:27:04.240747 12150 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260811 00:27:04.244779 12165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:04.247017 12150 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:27:04.247226 12150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "588271aba01b4dceb9f161bb519aa169"
format_stamp: "Formatted at 2026-08-11 00:27:04 on dist-test-slave-fd7l"
server_key: "3d51fb8dc1909c88b6e23894f87645cd"
server_key_iv: "6c2d6e8b7db1c87471b89b504878e081"
server_key_version: "encryptionkey@0"
I20260811 00:27:04.247464 12150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:04.264998 12150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:04.270304 12150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:04.270668 12150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:04.271715 12150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:27:04.273214 12150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:27:04.273325 12150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:04.273483 12150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:27:04.273576 12150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:04.310817 12150 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:43241
I20260811 00:27:04.310914 12294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:43241 every 8 connection(s)
I20260811 00:27:04.312263 12150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:27:04.318631 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 12150
I20260811 00:27:04.318845 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:27:04.319382 2528 external_mini_cluster.cc:1428] Setting key 177bd1a7ebbab6a29cc812bed25c6fe7
Aug 11 00:27:04 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408024, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:27:04.339480 12297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:04.315483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:51073 (local address 127.2.120.62:37663)
0811 00:27:04.316269 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:04.316277 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:04.317480 (+ 1203us) server_negotiation.cc:407] Connection header received
0811 00:27:04.319529 (+ 2049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:04.319538 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:04.319662 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:04.319797 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:04.322081 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:04.323231 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:04.324978 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:04.325862 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:04.326017 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:04.331384 (+ 5367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:04.331396 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:04.331403 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:04.331464 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:04.335438 (+ 3974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:04.336501 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:04.336571 (+ 70us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:04.336578 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:04.336738 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:04.337315 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:04.337321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:04.337323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:04.337670 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:04.337869 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:04.338736 (+ 867us) server_negotiation.cc:299] Negotiation successful
0811 00:27:04.339041 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":417,"threads_started":1}
I20260811 00:27:04.340724 12298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:04.316267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:51073)
0811 00:27:04.317263 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:04.317314 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:04.319192 (+ 1878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:04.320099 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:04.320112 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:04.320989 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:04.321887 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:04.321916 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:04.323428 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:04.323432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:04.324823 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:04.324836 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:04.326154 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:04.326164 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:04.326516 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:04.327434 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:04.327474 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:04.330984 (+ 3510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:27:04.335713 (+ 4729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:27:04.335724 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:27:04.335742 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:27:04.336312 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:27:04.336883 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:27:04.336889 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:27:04.336893 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:27:04.337157 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:27:04.337835 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:27:04.337844 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:27:04.338299 (+ 455us) client_negotiation.cc:769] Sending connection context
0811 00:27:04.338625 (+ 326us) client_negotiation.cc:240] Negotiation successful
0811 00:27:04.339302 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":427,"threads_started":1}
I20260811 00:27:04.342516 12295 heartbeater.cc:347] Connected to a master server at 127.2.120.62:37663
I20260811 00:27:04.342903 12295 heartbeater.cc:464] Registering TS with master...
I20260811 00:27:04.344007 12295 heartbeater.cc:511] Master 127.2.120.62:37663 requested a full tablet report, sending...
I20260811 00:27:04.346230 11941 ts_manager.cc:194] Registered new tserver with Master: 588271aba01b4dceb9f161bb519aa169 (127.2.120.2:43241)
I20260811 00:27:04.347584 11941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:51073
WARNING: no policy specified for kudu/127.2.120.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
I20260811 00:27:04.385061 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:37663
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:37203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:27:04.771776 12144 heartbeater.cc:503] Master 127.2.120.62:37663 was elected leader, sending a full tablet report...
W20260811 00:27:04.788414 12302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:04.789016 12302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:04.789149 12302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:04.798084 12302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:04.798244 12302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:04.798378 12302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:27:04.809926 12302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:37663
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:04.811399 12302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:04.812916 12302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:04.829741 12309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:04.829807 12311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:04.829782 12308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:04.830901 12302 server_base.cc:1034] running on GCE node
I20260811 00:27:04.831647 12302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:04.832909 12302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:04.834185 12302 hybrid_clock.cc:648] HybridClock initialized: now 1786408024834137 us; error 43 us; skew 500 ppm
I20260811 00:27:04.834607 12302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:27:04 dist-test-slave-fd7l krb5kdc[11889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408024, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:27:04.838886 12302 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:27:04.841096 12302 webserver.cc:460] Webserver started at http://127.2.120.3:34619/ using document root <none> and password file <none>
I20260811 00:27:04.841956 12302 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:04.842121 12302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:04.842546 12302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:27:04.847605 12302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "c0f618fcc28645eb81d59b04f9dda27b"
format_stamp: "Formatted at 2026-08-11 00:27:04 on dist-test-slave-fd7l"
server_key: "04d892d0765e4ac1f48a9a74b8e60e86"
server_key_iv: "cb69d2815ae3d22cd048d79bf139c84a"
server_key_version: "encryptionkey@0"
I20260811 00:27:04.848908 12302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "c0f618fcc28645eb81d59b04f9dda27b"
format_stamp: "Formatted at 2026-08-11 00:27:04 on dist-test-slave-fd7l"
server_key: "04d892d0765e4ac1f48a9a74b8e60e86"
server_key_iv: "cb69d2815ae3d22cd048d79bf139c84a"
server_key_version: "encryptionkey@0"
I20260811 00:27:04.855880 12302 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 00:27:04.860689 12321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:04.862736 12302 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260811 00:27:04.862998 12302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "c0f618fcc28645eb81d59b04f9dda27b"
format_stamp: "Formatted at 2026-08-11 00:27:04 on dist-test-slave-fd7l"
server_key: "04d892d0765e4ac1f48a9a74b8e60e86"
server_key_iv: "cb69d2815ae3d22cd048d79bf139c84a"
server_key_version: "encryptionkey@0"
I20260811 00:27:04.863476 12302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:04.887733 12302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:04.893646 12302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:04.894038 12302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:04.895339 12302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:27:04.897120 12302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:27:04.897234 12302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:04.897408 12302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:27:04.897501 12302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:04.935993 12302 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:39301
I20260811 00:27:04.936009 12446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:39301 every 8 connection(s)
I20260811 00:27:04.937848 12302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:27:04.939822 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 12302
I20260811 00:27:04.939985 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:27:04.940498 2528 external_mini_cluster.cc:1428] Setting key 2ef2b8fa5c7460ebdea0b05e92cc24ac
Aug 11 00:27:04 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408024, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:27:04.966504 12450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:04.941676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:57263 (local address 127.2.120.62:37663)
0811 00:27:04.943079 (+ 1403us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:04.943088 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:04.944720 (+ 1632us) server_negotiation.cc:407] Connection header received
0811 00:27:04.946172 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:04.946177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:04.946274 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:04.946389 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:04.948343 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:04.949328 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:04.951381 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:04.952304 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:04.952578 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:04.958389 (+ 5811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:04.958396 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:04.958400 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:04.958490 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:04.962436 (+ 3946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:04.963544 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:04.963552 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:04.963555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:04.963715 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:04.964246 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:04.964252 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:04.964255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:04.964695 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:04.964885 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:04.965768 (+ 883us) server_negotiation.cc:299] Negotiation successful
0811 00:27:04.966050 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1208,"thread_start_us":652,"threads_started":1}
I20260811 00:27:04.967639 12451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:04.942423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.3:57263)
0811 00:27:04.943508 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:04.943575 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:04.945794 (+ 2219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:04.946671 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:04.946681 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:04.947378 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:04.948099 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:04.948122 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:04.949527 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:04.949532 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:04.951014 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:04.951037 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:04.952818 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:04.952831 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:04.953237 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:04.954306 (+ 1069us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:04.954347 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:04.958059 (+ 3712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:27:04.962696 (+ 4637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:27:04.962718 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:27:04.962741 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:27:04.963375 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:27:04.963852 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:27:04.963857 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:27:04.963861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:27:04.964105 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:27:04.964847 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:27:04.964856 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:27:04.965316 (+ 460us) client_negotiation.cc:769] Sending connection context
0811 00:27:04.965650 (+ 334us) client_negotiation.cc:240] Negotiation successful
0811 00:27:04.966354 (+ 704us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"thread_start_us":451,"threads_started":1}
I20260811 00:27:04.969792 12447 heartbeater.cc:347] Connected to a master server at 127.2.120.62:37663
I20260811 00:27:04.970177 12447 heartbeater.cc:464] Registering TS with master...
I20260811 00:27:04.971195 12447 heartbeater.cc:511] Master 127.2.120.62:37663 requested a full tablet report, sending...
I20260811 00:27:04.973079 11941 ts_manager.cc:194] Registered new tserver with Master: c0f618fcc28645eb81d59b04f9dda27b (127.2.120.3:39301)
I20260811 00:27:04.974083 11941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:57263
I20260811 00:27:04.985671 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.1.keytab.
Entry for principal oryx/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.1.keytab.
WARNING: no policy specified for oryx/127.2.120.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.120.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.2.keytab.
Entry for principal oryx/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.2.keytab.
WARNING: no policy specified for oryx/127.2.120.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.120.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.3.keytab.
Entry for principal oryx/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.3.keytab.
I20260811 00:27:05.165306 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 12004
Aug 11 00:27:05 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:27:05.185978 12460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:05.166710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41358 (local address 127.2.120.1:40441)
0811 00:27:05.167484 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:05.167492 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:05.167525 (+ 33us) server_negotiation.cc:407] Connection header received
0811 00:27:05.167598 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:05.167606 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:05.167858 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:05.168029 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:05.169882 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:05.170956 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:05.172885 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:05.173571 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:05.173746 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:05.178938 (+ 5192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:05.178959 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:05.178979 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:05.179028 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:05.182402 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:05.183384 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:05.183394 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:05.183396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:05.183567 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:05.184060 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:05.184065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:05.184068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:05.184627 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:05.184924 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:05.185229 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 00:27:05.185484 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":400,"threads_started":1}
I20260811 00:27:05.187492 12138 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:27:05.277743 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 12004
I20260811 00:27:05.303802 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 12150
Aug 11 00:27:05 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.2@KRBTEST.COM
I20260811 00:27:05.330379 12462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:05.305124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40304 (local address 127.2.120.2:43241)
0811 00:27:05.306117 (+ 993us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:05.306129 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:05.306194 (+ 65us) server_negotiation.cc:407] Connection header received
0811 00:27:05.306542 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:05.306556 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:05.306878 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:05.307076 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:05.309773 (+ 2697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:05.311588 (+ 1815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:05.314088 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:05.315139 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:05.315315 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:05.322446 (+ 7131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:05.322466 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:05.322484 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:05.322539 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:05.326765 (+ 4226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:05.327746 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:05.327755 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:05.327757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:05.327933 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:05.328406 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:05.328411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:05.328413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:05.328910 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:05.329274 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:05.329658 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 00:27:05.329974 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":668,"threads_started":1}
I20260811 00:27:05.332001 12290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:27:05.419864 12295 heartbeater.cc:503] Master 127.2.120.62:37663 was elected leader, sending a full tablet report...
I20260811 00:27:05.421283 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 12150
W20260811 00:27:05.443020 11923 connection.cc:535] server connection from 127.2.120.2:51073 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 00:27:05.443367 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 12302
Aug 11 00:27:05 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.3@KRBTEST.COM
I20260811 00:27:05.466603 12464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:05.444988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45938 (local address 127.2.120.3:39301)
0811 00:27:05.445668 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:05.445676 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:05.445710 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:27:05.445776 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:05.445783 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:05.446037 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:05.446282 (+ 245us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:05.447812 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:05.448946 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:05.450564 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:05.451230 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:05.451388 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:05.456442 (+ 5054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:05.456461 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:05.456480 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:05.456592 (+ 112us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:05.460243 (+ 3651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:05.461375 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:05.461385 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:05.461388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:05.461566 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:05.462054 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:05.462059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:05.462062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:05.462692 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:05.465171 (+ 2479us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:05.465614 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 00:27:05.466002 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":361,"threads_started":1}
I20260811 00:27:05.468633 12441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:27:05.627318 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 12302
I20260811 00:27:05.648737 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 11904
I20260811 00:27:05.650135 11971 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:27:05.760087 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 11904
I20260811 00:27:05.788940 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:37663
--webserver_interface=127.2.120.62
--webserver_port=46191
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:37203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:37663
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:27:06.179770 12468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:06.180127 12468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:06.180320 12468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:06.188117 12468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:27:06.188274 12468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:06.188372 12468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:06.188462 12468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:27:06.188552 12468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:27:06.199312 12468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:37663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:37663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=46191
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:06.200922 12468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:06.202567 12468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:06.215579 12475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:06.215621 12474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:06.216279 12477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:06.216390 12468 server_base.cc:1034] running on GCE node
I20260811 00:27:06.217334 12468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:06.218922 12468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:06.220181 12468 hybrid_clock.cc:648] HybridClock initialized: now 1786408026220137 us; error 71 us; skew 500 ppm
I20260811 00:27:06.220643 12468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:27:06 dist-test-slave-fd7l krb5kdc[11889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408026, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:27:06.224272 12468 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:27:06.226229 12468 webserver.cc:460] Webserver started at http://127.2.120.62:46191/ using document root <none> and password file <none>
I20260811 00:27:06.227056 12468 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:06.227244 12468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:06.234826 12468 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 00:27:06.238832 12486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:06.240790 12468 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:27:06.241009 12468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "1d98caae9dd64e0aa55876ee62e351bd"
format_stamp: "Formatted at 2026-08-11 00:27:02 on dist-test-slave-fd7l"
server_key: "c5f11b90c8b707c2088ee052e088c487"
server_key_iv: "7cdc9bd5ffa66b856ae1d16bbe3d30ae"
server_key_version: "encryptionkey@0"
I20260811 00:27:06.242179 12468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:06.272224 12468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:06.277417 12468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:06.277697 12468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:06.298492 12468 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:37663
I20260811 00:27:06.298533 12544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:37663 every 8 connection(s)
I20260811 00:27:06.300207 12468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:27:06.301276 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 12468
I20260811 00:27:06.302395 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:40441
--local_ip_for_outbound_sockets=127.2.120.1
--tserver_master_addrs=127.2.120.62:37663
--webserver_port=37669
--webserver_interface=127.2.120.1
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:37203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.120.1
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:27:06.311798 12545 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:27:06.315881 12545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd: Bootstrap starting.
I20260811 00:27:06.337981 12545 log.cc:824] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd: Log is configured to *not* fsync() on all Append() calls
I20260811 00:27:06.349813 12545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:27:06.350525 12545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd: Bootstrap complete.
I20260811 00:27:06.361320 12545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d98caae9dd64e0aa55876ee62e351bd" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37663 } }
I20260811 00:27:06.362421 12545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d98caae9dd64e0aa55876ee62e351bd, State: Initialized, Role: FOLLOWER
I20260811 00:27:06.363411 12545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d98caae9dd64e0aa55876ee62e351bd" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37663 } }
I20260811 00:27:06.363736 12545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:27:06.363902 12545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:27:06.364135 12545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [term 1 FOLLOWER]: Advancing to term 2
I20260811 00:27:06.369143 12545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d98caae9dd64e0aa55876ee62e351bd" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37663 } }
I20260811 00:27:06.369824 12545 leader_election.cc:302] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d98caae9dd64e0aa55876ee62e351bd; no voters:
I20260811 00:27:06.370535 12545 leader_election.cc:288] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 00:27:06.370783 12550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [term 2 FOLLOWER]: Leader election won for term 2
I20260811 00:27:06.372143 12550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [term 2 LEADER]: Becoming Leader. State: Replica: 1d98caae9dd64e0aa55876ee62e351bd, State: Running, Role: LEADER
I20260811 00:27:06.372859 12550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d98caae9dd64e0aa55876ee62e351bd" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37663 } }
I20260811 00:27:06.373073 12545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:27:06.377138 12551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1d98caae9dd64e0aa55876ee62e351bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d98caae9dd64e0aa55876ee62e351bd" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37663 } } }
I20260811 00:27:06.377610 12551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [sys.catalog]: This master's current role is: LEADER
I20260811 00:27:06.378096 12552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d98caae9dd64e0aa55876ee62e351bd. Latest consensus state: current_term: 2 leader_uuid: "1d98caae9dd64e0aa55876ee62e351bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d98caae9dd64e0aa55876ee62e351bd" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37663 } } }
I20260811 00:27:06.378477 12552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd [sys.catalog]: This master's current role is: LEADER
I20260811 00:27:06.389864 12560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:27:06.398375 12560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:27:06.400322 12560 catalog_manager.cc:1259] Loaded cluster ID: cb3415432fc542e1be0d71b816f870cc
I20260811 00:27:06.400466 12560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:27:06.405941 12560 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:27:06.408119 12560 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1d98caae9dd64e0aa55876ee62e351bd: Loaded TSK: 0
I20260811 00:27:06.410429 12560 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 00:27:06.736778 12547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:06.737107 12547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:06.737267 12547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:06.745239 12547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:06.745371 12547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:06.745559 12547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:27:06.756798 12547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.1.keytab
--principal=oryx/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:40441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=37669
--tserver_master_addrs=127.2.120.62:37663
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:06.758359 12547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:06.759869 12547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:06.777889 12574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:06.777997 12576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:06.779107 12578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:06.780480 12547 server_base.cc:1034] running on GCE node
I20260811 00:27:06.781369 12547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:06.782621 12547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:06.783947 12547 hybrid_clock.cc:648] HybridClock initialized: now 1786408026783869 us; error 68 us; skew 500 ppm
I20260811 00:27:06.784438 12547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:27:06 dist-test-slave-fd7l krb5kdc[11889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408026, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:27:06.788429 12547 init.cc:377] Logged in from keytab as oryx/127.2.120.1@KRBTEST.COM (short username oryx)
I20260811 00:27:06.790675 12547 webserver.cc:460] Webserver started at http://127.2.120.1:37669/ using document root <none> and password file <none>
I20260811 00:27:06.791626 12547 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:06.791828 12547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:06.798645 12547 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 00:27:06.802453 12585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:06.804420 12547 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:27:06.804737 12547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "70095e40e16249bc859761aa0dc1c0e2"
format_stamp: "Formatted at 2026-08-11 00:27:03 on dist-test-slave-fd7l"
server_key: "07973a9544d4dd075da398f2884df8b6"
server_key_iv: "d385b3d41992fafdc526d3ca52bf6d05"
server_key_version: "encryptionkey@0"
I20260811 00:27:06.805938 12547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:06.823326 12547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:06.828675 12547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:06.828979 12547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:06.830072 12547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:27:06.831665 12547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:27:06.831769 12547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:06.831866 12547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:27:06.831948 12547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:06.871752 12547 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:40441
I20260811 00:27:06.871852 12711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:40441 every 8 connection(s)
I20260811 00:27:06.873595 12547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:27:06.877152 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 12547
I20260811 00:27:06.877935 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:43241
--local_ip_for_outbound_sockets=127.2.120.2
--tserver_master_addrs=127.2.120.62:37663
--webserver_port=33637
--webserver_interface=127.2.120.2
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:37203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.120.2
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:27:06 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:06 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 00:27:06.897639 12715 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:06.877135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:41249 (local address 127.2.120.62:37663)
0811 00:27:06.878002 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:06.878026 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:06.878925 (+ 899us) server_negotiation.cc:407] Connection header received
0811 00:27:06.881258 (+ 2333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:06.881282 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:06.881763 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:06.882243 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:06.883792 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:06.886085 (+ 2293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:06.887848 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:06.889042 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:06.889375 (+ 333us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:06.896609 (+ 7234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.1:41249: BlockingRecv error: recv got EOF from 127.2.120.1:41249 (error 108)
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":296,"threads_started":1}
I20260811 00:27:06.897692 12716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:06.877304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:41249)
0811 00:27:06.878677 (+ 1373us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:06.878737 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:06.880590 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:06.882338 (+ 1748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:06.882346 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:06.882935 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:06.883513 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:06.883554 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:06.886293 (+ 2739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:06.886298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:06.887651 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:06.887668 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:06.889627 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:06.889636 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:06.889886 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:06.890779 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:06.890808 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:06.896077 (+ 5269us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1057,"thread_start_us":616,"threads_started":1}
W20260811 00:27:06.898627 12713 heartbeater.cc:644] Failed to heartbeat to 127.2.120.62:37663 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.120.62:37663: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Aug 11 00:27:06 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:06 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:06.909654 12716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:06.899479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:59623)
0811 00:27:06.899702 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:06.899725 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:06.899861 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:06.900353 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:06.900359 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:06.900812 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:06.900989 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:06.900999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:06.902821 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:06.902825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:06.904017 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:06.904033 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:06.905247 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:06.905252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:06.905401 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:06.906161 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:06.906180 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:06.909379 (+ 3199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 00:27:06.910485 12715 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:06.899685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:59623 (local address 127.2.120.62:37663)
0811 00:27:06.899908 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:06.899916 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:06.899952 (+ 36us) server_negotiation.cc:407] Connection header received
0811 00:27:06.900024 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:06.900032 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:06.900124 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:06.900245 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:06.901173 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:06.902542 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:06.904203 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:06.905044 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:06.905215 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:06.909737 (+ 4522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.1:59623: BlockingRecv error: recv got EOF from 127.2.120.1:59623 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 00:27:06 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:06 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:06.920320 12716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:06.910710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:53867)
0811 00:27:06.910938 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:06.910955 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:06.911066 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:06.911609 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:06.911615 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:06.912077 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:06.912268 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:06.912279 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:06.913556 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:06.913564 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:06.914819 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:06.914835 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:06.915670 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:06.915676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:06.915912 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:06.916847 (+ 935us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:06.916867 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:06.919958 (+ 3091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 00:27:06.920477 12715 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:06.911052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:53867 (local address 127.2.120.62:37663)
0811 00:27:06.911282 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:06.911287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:06.911303 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:27:06.911357 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:06.911361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:06.911421 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:06.911553 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:06.912440 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:06.913375 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:06.915041 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:06.915489 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:06.915639 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:06.920225 (+ 4586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.1:53867: BlockingRecv error: recv got EOF from 127.2.120.1:53867 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 00:27:06.920950 12713 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:27:07.279654 12717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:07.280105 12717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:07.280346 12717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:07.290227 12717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:07.290385 12717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:07.290545 12717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:27:07.302196 12717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.2.keytab
--principal=oryx/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:43241
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=33637
--tserver_master_addrs=127.2.120.62:37663
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:07.304068 12717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:07.305917 12717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:07.323061 12723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:07.323282 12726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:07.323184 12724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:07.325613 12717 server_base.cc:1034] running on GCE node
I20260811 00:27:07.326956 12717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:07.328675 12717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:07.330025 12717 hybrid_clock.cc:648] HybridClock initialized: now 1786408027329944 us; error 76 us; skew 500 ppm
I20260811 00:27:07.330503 12717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:27:07 dist-test-slave-fd7l krb5kdc[11889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408027, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:27:07.334527 12717 init.cc:377] Logged in from keytab as oryx/127.2.120.2@KRBTEST.COM (short username oryx)
I20260811 00:27:07.336483 12717 webserver.cc:460] Webserver started at http://127.2.120.2:33637/ using document root <none> and password file <none>
I20260811 00:27:07.337419 12717 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:07.337621 12717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:07.344020 12717 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.002s
I20260811 00:27:07.347265 12733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:07.348949 12717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:27:07.349174 12717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "588271aba01b4dceb9f161bb519aa169"
format_stamp: "Formatted at 2026-08-11 00:27:04 on dist-test-slave-fd7l"
server_key: "3d51fb8dc1909c88b6e23894f87645cd"
server_key_iv: "6c2d6e8b7db1c87471b89b504878e081"
server_key_version: "encryptionkey@0"
I20260811 00:27:07.350270 12717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:07.366580 12717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:07.371840 12717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:07.372198 12717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:07.373639 12717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:27:07.375448 12717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:27:07.375597 12717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:07.375753 12717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:27:07.375851 12717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:07.417554 12717 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:43241
I20260811 00:27:07.417613 12859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:43241 every 8 connection(s)
I20260811 00:27:07.419178 12717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:27:07.424621 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 12717
I20260811 00:27:07.425483 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:39301
--local_ip_for_outbound_sockets=127.2.120.3
--tserver_master_addrs=127.2.120.62:37663
--webserver_port=34619
--webserver_interface=127.2.120.3
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:37203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.120.3
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:27:07 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:07 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 00:27:07.446870 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:07.422811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:49241 (local address 127.2.120.62:37663)
0811 00:27:07.423471 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:07.423476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:07.424940 (+ 1464us) server_negotiation.cc:407] Connection header received
0811 00:27:07.428333 (+ 3393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:07.428340 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:07.428462 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:07.428754 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:07.431638 (+ 2884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:07.432502 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:07.436437 (+ 3935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:07.437312 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:07.437553 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:07.446269 (+ 8716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.2:49241: BlockingRecv error: recv got EOF from 127.2.120.2:49241 (error 108)
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":387,"threads_started":1}
I20260811 00:27:07.447093 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:07.423658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:49241)
0811 00:27:07.424716 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:07.424784 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:07.427750 (+ 2966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:07.429156 (+ 1406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:07.429173 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:07.430193 (+ 1020us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:07.431263 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:07.431300 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:07.432817 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:07.432826 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:07.435991 (+ 3165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:07.436039 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:07.437781 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:07.437791 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:07.438134 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:07.439279 (+ 1145us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:07.439329 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:07.445606 (+ 6277us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":389,"threads_started":1}
W20260811 00:27:07.448125 12860 heartbeater.cc:644] Failed to heartbeat to 127.2.120.62:37663 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.120.62:37663: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Aug 11 00:27:07 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:07 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:07.462101 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:07.448864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:59363)
0811 00:27:07.449139 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:07.449275 (+ 136us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:07.449432 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:07.449820 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:07.449827 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:07.450455 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:07.450714 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:07.450730 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:07.452070 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:07.452079 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:07.453887 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:07.453912 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:07.455688 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:07.455698 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:07.455978 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:07.456887 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:07.456920 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:07.461692 (+ 4772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":82}
W20260811 00:27:07.462337 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:07.449126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:59363 (local address 127.2.120.62:37663)
0811 00:27:07.449312 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:07.449317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:07.449340 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:27:07.449583 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:07.449587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:07.449661 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:07.449751 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:07.451028 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:07.451872 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:07.454327 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:07.455171 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:07.455429 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:07.462034 (+ 6605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.2:59363: BlockingRecv error: recv got EOF from 127.2.120.2:59363 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 00:27:07 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:07 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:07.474984 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:07.463579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:47683)
0811 00:27:07.463936 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:07.463956 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:07.464136 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:07.464830 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:07.464839 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:07.465341 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:07.465615 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:07.465632 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:07.467468 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:07.467474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:07.469107 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:07.469124 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:07.469889 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:07.469895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:07.470108 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:07.470767 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:07.470785 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:07.474413 (+ 3628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190}
W20260811 00:27:07.475234 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:07.463512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:47683 (local address 127.2.120.62:37663)
0811 00:27:07.463753 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:07.463761 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:07.464062 (+ 301us) server_negotiation.cc:407] Connection header received
0811 00:27:07.464306 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:07.464314 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:07.464439 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:07.464622 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:07.465801 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:07.467286 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:07.469283 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:07.469723 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:07.469864 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:07.474700 (+ 4836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.2:47683: BlockingRecv error: recv got EOF from 127.2.120.2:47683 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 00:27:07.475679 12860 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:27:07.827419 12864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:07.827762 12864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:07.827860 12864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:07.835669 12864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:07.835813 12864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:07.835933 12864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:27:07.846748 12864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/krb5kdc/oryx_127.2.120.3.keytab
--principal=oryx/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:39301
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=34619
--tserver_master_addrs=127.2.120.62:37663
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:07.848261 12864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:07.850001 12864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:07.864439 12869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:07.867539 12872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:07.867689 12870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:07.867818 12864 server_base.cc:1034] running on GCE node
I20260811 00:27:07.868772 12864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:07.869935 12864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:07.871285 12864 hybrid_clock.cc:648] HybridClock initialized: now 1786408027871150 us; error 75 us; skew 500 ppm
I20260811 00:27:07.871714 12864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:27:07 dist-test-slave-fd7l krb5kdc[11889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408027, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:27:07.875741 12864 init.cc:377] Logged in from keytab as oryx/127.2.120.3@KRBTEST.COM (short username oryx)
I20260811 00:27:07.878790 12864 webserver.cc:460] Webserver started at http://127.2.120.3:34619/ using document root <none> and password file <none>
I20260811 00:27:07.879693 12864 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:07.879897 12864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:07.887285 12864 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 00:27:07.891536 12879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:07.893664 12864 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 00:27:07.893919 12864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "c0f618fcc28645eb81d59b04f9dda27b"
format_stamp: "Formatted at 2026-08-11 00:27:04 on dist-test-slave-fd7l"
server_key: "04d892d0765e4ac1f48a9a74b8e60e86"
server_key_iv: "cb69d2815ae3d22cd048d79bf139c84a"
server_key_version: "encryptionkey@0"
I20260811 00:27:07.895081 12864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:07.922508 12864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:07.928740 12864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:07.929132 12864 kserver.cc:163] Server-wide thread pool size limit: 3276
Aug 11 00:27:07 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:07.930781 12864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
Aug 11 00:27:07 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:07.933096 12864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:27:07.933158 12885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:07.922019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:55745)
0811 00:27:07.922790 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:07.922818 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:07.923006 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:07.923389 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:07.923394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:07.923921 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:07.924136 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:07.924146 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:07.925363 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:07.925369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:07.926445 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:07.926461 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:07.927361 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:07.927367 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:07.927570 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:07.928223 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:07.928243 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:07.932785 (+ 4542us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":452,"threads_started":1}
I20260811 00:27:07.933249 12864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:07.933368 12864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:27:07.933493 12864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
W20260811 00:27:07.933704 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:07.922236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:55745 (local address 127.2.120.62:37663)
0811 00:27:07.922506 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:07.922515 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:07.922919 (+ 404us) server_negotiation.cc:407] Connection header received
0811 00:27:07.923166 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:07.923170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:07.923240 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:07.923353 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:07.924306 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:07.925190 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:07.926678 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:07.927140 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:07.927298 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:07.933356 (+ 6058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.1:55745: BlockingRecv error: recv got EOF from 127.2.120.1:55745 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:27:07.984961 12864 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:39301
I20260811 00:27:07.985016 13005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:39301 every 8 connection(s)
I20260811 00:27:07.987501 12864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:27:07.998364 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 12864
Aug 11 00:27:08 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:08 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 00:27:08.017675 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:07.992902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:59127 (local address 127.2.120.62:37663)
0811 00:27:07.993179 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:07.993185 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:07.995730 (+ 2545us) server_negotiation.cc:407] Connection header received
0811 00:27:07.998477 (+ 2747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:07.998484 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:07.998688 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:07.998866 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:08.003435 (+ 4569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.004420 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.007544 (+ 3124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.008339 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.008525 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:08.016820 (+ 8295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.3:59127: BlockingRecv error: recv got EOF from 127.2.120.3:59127 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 00:27:08.018239 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:07.993788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.3:59127)
0811 00:27:07.995364 (+ 1576us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:07.995454 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:07.997932 (+ 2478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:07.999355 (+ 1423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:07.999373 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:08.000876 (+ 1503us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:08.003034 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:08.003105 (+ 71us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.004816 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.004823 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:08.007333 (+ 2510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:08.007356 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.008781 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.008800 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:08.009137 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:08.010304 (+ 1167us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:08.010347 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:08.016341 (+ 5994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1072,"thread_start_us":680,"threads_started":1}
W20260811 00:27:08.019497 13006 heartbeater.cc:644] Failed to heartbeat to 127.2.120.62:37663 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.120.62:37663: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
I20260811 00:27:08.024848 13011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:08.000459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53988 (local address 127.2.120.62:37663)
0811 00:27:08.001801 (+ 1342us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:08.001807 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:08.001840 (+ 33us) server_negotiation.cc:407] Connection header received
0811 00:27:08.002067 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:08.002076 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:08.002452 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:08.002865 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:08.005413 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.006821 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.008865 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.009398 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.009637 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:08.014589 (+ 4952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:08.014604 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:08.014638 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:08.014716 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:08.019877 (+ 5161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:08.021077 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:08.021086 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:08.021094 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:08.021378 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:08.021862 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:08.021867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:08.021869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:08.022436 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:08.022988 (+ 552us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:08.023923 (+ 935us) server_negotiation.cc:299] Negotiation successful
0811 00:27:08.024429 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1133,"thread_start_us":878,"threads_started":1}
Aug 11 00:27:08 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:08 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:08.033069 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:08.020353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.3:35341)
0811 00:27:08.020793 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:08.020827 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:08.021072 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:08.021676 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:08.021681 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:08.022091 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:08.022271 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:08.022281 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.023496 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.023504 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:08.024871 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:08.024897 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.028036 (+ 3139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.028043 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:08.028235 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:08.029099 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:08.029129 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:08.032817 (+ 3688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198}
W20260811 00:27:08.033403 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:08.020774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:35341 (local address 127.2.120.62:37663)
0811 00:27:08.021271 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:08.021281 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:08.021326 (+ 45us) server_negotiation.cc:407] Connection header received
0811 00:27:08.021394 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:08.021400 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:08.021502 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:08.021639 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:08.022442 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.023333 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.026881 (+ 3548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.027730 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.027972 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:08.033123 (+ 5151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.3:35341: BlockingRecv error: recv got EOF from 127.2.120.3:35341 (error 108)
Metrics: {"server-negotiator.queue_time_us":229}
Aug 11 00:27:08 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:08 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:08.043542 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:08.034028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.3:35813)
0811 00:27:08.034215 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:08.034243 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:08.034393 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:08.034767 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:08.034772 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:08.035145 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:08.035319 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:08.035329 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.036415 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.036421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:08.037475 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:08.037490 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.038297 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.038303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:08.038535 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:08.039200 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:08.039218 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:08.043183 (+ 3965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 00:27:08.043762 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:08.034219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:35813 (local address 127.2.120.62:37663)
0811 00:27:08.034458 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:08.034464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:08.034486 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:27:08.034548 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:08.034553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:08.034626 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:08.034738 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:08.035480 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.036242 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.037654 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.038124 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.038277 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:08.043456 (+ 5179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.3:35813: BlockingRecv error: recv got EOF from 127.2.120.3:35813 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260811 00:27:08.044126 13006 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 00:27:08 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:08 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:08.486485 13012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:08.476783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:47359)
0811 00:27:08.477525 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:08.477545 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:08.477736 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:08.478094 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:08.478099 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:08.478632 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:08.478865 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:08.478875 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.479995 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.480000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:08.481054 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:08.481067 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.481863 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.481869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:08.482063 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:08.482795 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:08.482813 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:08.486110 (+ 3297us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":402,"threads_started":1}
W20260811 00:27:08.486583 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:08.476971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:47359 (local address 127.2.120.62:37663)
0811 00:27:08.477284 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:08.477293 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:08.477650 (+ 357us) server_negotiation.cc:407] Connection header received
0811 00:27:08.477871 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:08.477884 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:08.477948 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:08.478057 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:08.479033 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.479829 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.481214 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.481659 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.481800 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:08.486336 (+ 4536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.2:47359: BlockingRecv error: recv got EOF from 127.2.120.2:47359 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 11 00:27:08 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:08 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:08.944118 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:08.934360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:39511)
0811 00:27:08.934982 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:08.935000 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:08.935163 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:08.935615 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:08.935619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:08.936059 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:08.936225 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:08.936233 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.937442 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.937446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:08.938539 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:08.938558 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.939369 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.939374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:08.939497 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:08.940267 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:08.940288 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:08.943717 (+ 3429us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":358,"threads_started":1}
W20260811 00:27:08.944257 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:08.934442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:39511 (local address 127.2.120.62:37663)
0811 00:27:08.934677 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:08.934683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:08.935066 (+ 383us) server_negotiation.cc:407] Connection header received
0811 00:27:08.935327 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:08.935331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:08.935413 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:08.935507 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:08.936416 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.937235 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.938739 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:08.939198 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:08.939310 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:08.943996 (+ 4686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.1:39511: BlockingRecv error: recv got EOF from 127.2.120.1:39511 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 00:27:09 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:09 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:09.054610 13015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:09.044941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.3:37027)
0811 00:27:09.045584 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:09.045604 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:09.045795 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:09.046169 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:09.046173 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:09.046716 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:09.046968 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:09.046978 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:09.048095 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:09.048104 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:09.049195 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:09.049209 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:09.050016 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:09.050024 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:09.050200 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:09.050974 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:09.050996 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:09.054351 (+ 3355us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":370,"threads_started":1}
W20260811 00:27:09.055235 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:09.045246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:37027 (local address 127.2.120.62:37663)
0811 00:27:09.045544 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:09.045551 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:09.045706 (+ 155us) server_negotiation.cc:407] Connection header received
0811 00:27:09.045932 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:09.045937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:09.046013 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:09.046136 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:09.047133 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:09.047933 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:09.049353 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:09.049795 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:09.049939 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:09.054695 (+ 4756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.3:37027: BlockingRecv error: recv got EOF from 127.2.120.3:37027 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 11 00:27:09 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:09 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:09.498018 13016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:09.487810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:38699)
0811 00:27:09.488560 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:09.488587 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:09.488774 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:09.489145 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:09.489150 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:09.489627 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:09.489779 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:09.489788 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:09.490957 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:09.490971 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:09.491908 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:09.491921 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:09.492817 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:09.492823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:09.493015 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:09.493860 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:09.493879 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:09.497745 (+ 3866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":420,"threads_started":1}
W20260811 00:27:09.498445 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:09.488076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:38699 (local address 127.2.120.62:37663)
0811 00:27:09.488376 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:09.488383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:09.488688 (+ 305us) server_negotiation.cc:407] Connection header received
0811 00:27:09.488908 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:09.488913 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:09.488993 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:09.489108 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:09.489981 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:09.490794 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:09.492084 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:09.492639 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:09.492787 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:09.498016 (+ 5229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.2:38699: BlockingRecv error: recv got EOF from 127.2.120.2:38699 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 11 00:27:09 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:09 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:09.955273 13018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:09.945380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:38991)
0811 00:27:09.945997 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:09.946014 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:09.946155 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:09.946497 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:09.946500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:09.946918 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:09.947113 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:09.947123 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:09.948369 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:09.948374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:09.949375 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:09.949390 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:09.950147 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:09.950151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:09.950290 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:09.951213 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:09.951233 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:09.954887 (+ 3654us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":338,"threads_started":1}
W20260811 00:27:09.955483 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:09.945605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:38991 (local address 127.2.120.62:37663)
0811 00:27:09.945803 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:09.945812 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:09.946142 (+ 330us) server_negotiation.cc:407] Connection header received
0811 00:27:09.946275 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:09.946280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:09.946354 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:09.946461 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:09.947269 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:09.948207 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:09.949549 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:09.950005 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:09.950114 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:09.955192 (+ 5078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.1:38991: BlockingRecv error: recv got EOF from 127.2.120.1:38991 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 00:27:10 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:10 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:10.067194 13019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:10.055851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.3:46145)
0811 00:27:10.056845 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:10.056869 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:10.057162 (+ 293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:10.057835 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:10.057839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:10.058414 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:10.058630 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:10.058642 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:10.059924 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:10.059930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:10.061234 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:10.061262 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:10.062271 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:10.062277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:10.062406 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:10.063324 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:10.063347 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:10.066911 (+ 3564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":866,"thread_start_us":664,"threads_started":1}
W20260811 00:27:10.067847 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:10.056005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:46145 (local address 127.2.120.62:37663)
0811 00:27:10.056594 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:10.056604 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:10.057054 (+ 450us) server_negotiation.cc:407] Connection header received
0811 00:27:10.057493 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:10.057498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:10.057595 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:10.057729 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:10.058838 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:10.059645 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:10.061472 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:10.062065 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:10.062208 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:10.067534 (+ 5326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.3:46145: BlockingRecv error: recv got EOF from 127.2.120.3:46145 (error 108)
Metrics: {"server-negotiator.queue_time_us":156}
Aug 11 00:27:10 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:10 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:10.508922 13020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:10.499607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:55991)
0811 00:27:10.500312 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:10.500329 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:10.500544 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:10.500910 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:10.500914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:10.501323 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:10.501508 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:10.501517 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:10.502607 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:10.502613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:10.503513 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:10.503526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:10.504297 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:10.504304 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:10.504543 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:10.505262 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:10.505282 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:10.508550 (+ 3268us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":373,"threads_started":1}
W20260811 00:27:10.509069 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:10.499687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:55991 (local address 127.2.120.62:37663)
0811 00:27:10.500022 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:10.500030 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:10.500427 (+ 397us) server_negotiation.cc:407] Connection header received
0811 00:27:10.500689 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:10.500694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:10.500758 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:10.500867 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:10.501693 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:10.502451 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:10.503671 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:10.504134 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:10.504277 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:10.508794 (+ 4517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.2:55991: BlockingRecv error: recv got EOF from 127.2.120.2:55991 (error 108)
Metrics: {"server-negotiator.queue_time_us":130}
Aug 11 00:27:10 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:10 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:10.967253 13021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:10.956651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:59487)
0811 00:27:10.957270 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:10.957287 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:10.957468 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:10.957991 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:10.957996 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:10.958570 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:10.958795 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:10.958811 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:10.960269 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:10.960277 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:10.961664 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:10.961684 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:10.962585 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:10.962590 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:10.962761 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:10.963515 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:10.963535 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:10.966988 (+ 3453us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":342,"threads_started":1}
W20260811 00:27:10.967540 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:10.956875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:59487 (local address 127.2.120.62:37663)
0811 00:27:10.957150 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:10.957157 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:10.957375 (+ 218us) server_negotiation.cc:407] Connection header received
0811 00:27:10.957618 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:10.957629 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:10.957730 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:10.957861 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:10.958979 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:10.959972 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:10.961849 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:10.962393 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:10.962522 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:10.967273 (+ 4751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.1:59487: BlockingRecv error: recv got EOF from 127.2.120.1:59487 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 11 00:27:11 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:11 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:11.078306 13022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:11.068727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.3:34245)
0811 00:27:11.069373 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:11.069391 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:11.069574 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:11.069934 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:11.069939 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:11.070351 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:11.070536 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:11.070546 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:11.071603 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:11.071614 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:11.072830 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:11.072846 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:11.073975 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:11.073985 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:11.074151 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:11.074902 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:11.074920 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:11.078030 (+ 3110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":301,"threads_started":1}
W20260811 00:27:11.078758 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:11.068854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:34245 (local address 127.2.120.62:37663)
0811 00:27:11.069130 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:11.069138 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:11.069493 (+ 355us) server_negotiation.cc:407] Connection header received
0811 00:27:11.069710 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:11.069720 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:11.069787 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:11.069898 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:11.070716 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:11.071429 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:11.073047 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:11.073767 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:11.073939 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:11.078319 (+ 4380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.3:34245: BlockingRecv error: recv got EOF from 127.2.120.3:34245 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 11 00:27:11 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:11 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:11.520483 13023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:11.510178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:60507)
0811 00:27:11.510747 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:11.510765 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:11.510919 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:11.511354 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:11.511358 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:11.511782 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:11.511942 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:11.511952 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:11.513355 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:11.513364 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:11.514562 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:11.514583 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:11.515544 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:11.515549 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:11.515751 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:11.516873 (+ 1122us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:11.516901 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:11.520194 (+ 3293us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":309,"threads_started":1}
W20260811 00:27:11.520828 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:11.510229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:60507 (local address 127.2.120.62:37663)
0811 00:27:11.510498 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:11.510503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:11.510842 (+ 339us) server_negotiation.cc:407] Connection header received
0811 00:27:11.511100 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:11.511106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:11.511196 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:11.511290 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:11.512125 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:11.513028 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:11.514818 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:11.515300 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:11.515436 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:11.520469 (+ 5033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.2:60507: BlockingRecv error: recv got EOF from 127.2.120.2:60507 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 11 00:27:11 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:11 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:11.977341 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:11.968414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:43771)
0811 00:27:11.969064 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:11.969080 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:11.969253 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:11.969636 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:11.969641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:11.970109 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:11.970270 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:11.970278 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:11.971493 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:11.971499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:11.972411 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:11.972423 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:11.973196 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:11.973202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:11.973374 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:11.973957 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:11.974051 (+ 94us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:11.976982 (+ 2931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":333,"threads_started":1}
W20260811 00:27:11.977465 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:11.968651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:43771 (local address 127.2.120.62:37663)
0811 00:27:11.968911 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:11.968917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:11.969178 (+ 261us) server_negotiation.cc:407] Connection header received
0811 00:27:11.969392 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:11.969398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:11.969483 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:11.969601 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:11.970422 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:11.971348 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:11.972584 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:11.973052 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:11.973178 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:11.977212 (+ 4034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.1:43771: BlockingRecv error: recv got EOF from 127.2.120.1:43771 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 00:27:12 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:12 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:12.091713 13025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:12.079472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.3:32965)
0811 00:27:12.080081 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:12.080106 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:12.080293 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:12.081097 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:12.081101 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:12.081531 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:12.081717 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:12.081726 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:12.083059 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:12.083063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:12.084140 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:12.084156 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:12.085227 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:12.085234 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:12.085399 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:12.086308 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:12.086337 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:12.091303 (+ 4966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":346,"threads_started":1}
W20260811 00:27:12.092042 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:12.079564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:32965 (local address 127.2.120.62:37663)
0811 00:27:12.079883 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:12.079891 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:12.080209 (+ 318us) server_negotiation.cc:407] Connection header received
0811 00:27:12.080730 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:12.080737 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:12.080865 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:12.081023 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:12.081931 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:12.082879 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:12.084331 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:12.084992 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:12.085144 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:12.091579 (+ 6435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.3:32965: BlockingRecv error: recv got EOF from 127.2.120.3:32965 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 11 00:27:12 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:12 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:12.532449 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:12.521929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:39731)
0811 00:27:12.522583 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:12.522601 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:12.522740 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:12.523196 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:12.523201 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:12.523801 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:12.524116 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:12.524130 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:12.525553 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:12.525558 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:12.526667 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:12.526683 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:12.527638 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:12.527643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:12.527789 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:12.528572 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:12.528592 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:12.532131 (+ 3539us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":346,"threads_started":1}
W20260811 00:27:12.532784 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:12.522288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:39731 (local address 127.2.120.62:37663)
0811 00:27:12.522481 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:12.522486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:12.522719 (+ 233us) server_negotiation.cc:407] Connection header received
0811 00:27:12.522866 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:12.522870 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:12.522945 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:12.523040 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:12.524386 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:12.525353 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:12.526918 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:12.527449 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:12.527587 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:12.532430 (+ 4843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.2:39731: BlockingRecv error: recv got EOF from 127.2.120.2:39731 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 00:27:12 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:12 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:12.994590 13027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:12.978891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:58617)
0811 00:27:12.979516 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:12.979533 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:12.979735 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:12.980497 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:12.980543 (+ 46us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:12.981343 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:12.981910 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:12.981940 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:12.983494 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:12.983499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:12.984724 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:12.984744 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:12.985633 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:12.985639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:12.985839 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:12.989989 (+ 4150us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:12.990015 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:12.994208 (+ 4193us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":359,"threads_started":1}
W20260811 00:27:12.994773 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:12.979044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:58617 (local address 127.2.120.62:37663)
0811 00:27:12.979341 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:12.979348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:12.979641 (+ 293us) server_negotiation.cc:407] Connection header received
0811 00:27:12.979979 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:12.979988 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:12.980128 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:12.980258 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:12.982145 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:12.983316 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:12.984925 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:12.985427 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:12.985557 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:12.994490 (+ 8933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.1:58617: BlockingRecv error: recv got EOF from 127.2.120.1:58617 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 11 00:27:13 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:13 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:13.102753 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:13.093092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.3:54187)
0811 00:27:13.093654 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:13.093671 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:13.093813 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:13.094155 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:13.094158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:13.094600 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:13.094774 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:13.094784 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:13.095944 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:13.095948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:13.096957 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:13.096971 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:13.098041 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:13.098048 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:13.098258 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:13.099083 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:13.099107 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:13.102499 (+ 3392us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":293,"threads_started":1}
W20260811 00:27:13.103041 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:13.093169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:54187 (local address 127.2.120.62:37663)
0811 00:27:13.093443 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:13.093451 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:13.093767 (+ 316us) server_negotiation.cc:407] Connection header received
0811 00:27:13.093929 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:13.093933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:13.094016 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:13.094117 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:13.094941 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:13.095794 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:13.097143 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:13.097822 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:13.097969 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:13.102747 (+ 4778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.3:54187: BlockingRecv error: recv got EOF from 127.2.120.3:54187 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 11 00:27:13 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:13 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:13.544273 13029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:13.533795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:41961)
0811 00:27:13.534535 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:13.534552 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:13.534725 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:13.535079 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:13.535084 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:13.535646 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:13.535827 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:13.535837 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:13.537511 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:13.537517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:13.538891 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:13.538921 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:13.540033 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:13.540039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:13.540183 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:13.540877 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:13.540897 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:13.544032 (+ 3135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":361,"threads_started":1}
W20260811 00:27:13.544711 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:13.534012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:41961 (local address 127.2.120.62:37663)
0811 00:27:13.534418 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:13.534427 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:13.534648 (+ 221us) server_negotiation.cc:407] Connection header received
0811 00:27:13.534858 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:13.534868 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:13.534939 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:13.535045 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:13.536013 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:13.537244 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:13.539167 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:13.539826 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:13.539943 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:13.544359 (+ 4416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.2:41961: BlockingRecv error: recv got EOF from 127.2.120.2:41961 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 11 00:27:14 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:14 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 00:27:14.008276 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:13.996188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:40885 (local address 127.2.120.62:37663)
0811 00:27:13.996627 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:13.996639 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:13.997428 (+ 789us) server_negotiation.cc:407] Connection header received
0811 00:27:13.997594 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:13.997604 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:13.997727 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:13.997868 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:13.999097 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:14.000453 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:14.002042 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:14.002524 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:14.002657 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:14.008033 (+ 5376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.1:40885: BlockingRecv error: recv got EOF from 127.2.120.1:40885 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
I20260811 00:27:14.008308 13031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:13.996012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:40885)
0811 00:27:13.997197 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:13.997220 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:13.997397 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:13.997995 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:13.998000 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:13.998597 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:13.998895 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:13.998914 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:14.000711 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:14.000716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:14.001829 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:14.001845 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:14.002740 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:14.002745 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:14.002893 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:14.003770 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:14.003791 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:14.007791 (+ 4000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1035,"thread_start_us":712,"threads_started":1}
Aug 11 00:27:14 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:14 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:14.113345 13032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:14.103926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.3:36429)
0811 00:27:14.104611 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:14.104629 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:14.104800 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:14.105155 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:14.105159 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:14.105742 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:14.105957 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:14.105969 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:14.107114 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:14.107121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:14.108042 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:14.108054 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:14.108864 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:14.108871 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:14.109057 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:14.109829 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:14.109848 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:14.113106 (+ 3258us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":372,"threads_started":1}
W20260811 00:27:14.113722 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:14.104079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:36429 (local address 127.2.120.62:37663)
0811 00:27:14.104374 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:14.104381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:14.104719 (+ 338us) server_negotiation.cc:407] Connection header received
0811 00:27:14.104932 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:14.104937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:14.105009 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:14.105125 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:14.106092 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:14.106964 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:14.108193 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:14.108687 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:14.108822 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:14.113433 (+ 4611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.3:36429: BlockingRecv error: recv got EOF from 127.2.120.3:36429 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 11 00:27:14 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:14 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:14.555743 13033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:14.545375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:52263)
0811 00:27:14.546054 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:14.546071 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:14.546251 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:14.546633 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:14.546638 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:14.547121 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:14.547357 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:14.547368 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:14.548459 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:14.548465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:14.549413 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:14.549425 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:14.550175 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:14.550181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:14.550352 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:14.551029 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:14.551048 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:14.555274 (+ 4226us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":310,"threads_started":1}
W20260811 00:27:14.555895 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:14.545541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:52263 (local address 127.2.120.62:37663)
0811 00:27:14.545810 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:14.545818 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:14.546186 (+ 368us) server_negotiation.cc:407] Connection header received
0811 00:27:14.546389 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:14.546399 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:14.546475 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:14.546605 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:14.547505 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:14.548302 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:14.549565 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:14.549985 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:14.550131 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:14.555557 (+ 5426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.2:52263: BlockingRecv error: recv got EOF from 127.2.120.2:52263 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
Aug 11 00:27:15 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:15 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:15.020236 13034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:15.009752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:60623)
0811 00:27:15.010459 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:15.010480 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:15.010631 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:15.011004 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:15.011007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:15.011526 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:15.011711 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:15.011721 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:15.013219 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:15.013226 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:15.014319 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:15.014334 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:15.015774 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:15.015780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:15.015933 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:15.016861 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:15.016881 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:15.019935 (+ 3054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":351,"threads_started":1}
W20260811 00:27:15.020630 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:15.010012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:60623 (local address 127.2.120.62:37663)
0811 00:27:15.010292 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:15.010301 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:15.010662 (+ 361us) server_negotiation.cc:407] Connection header received
0811 00:27:15.010756 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:15.010761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:15.010843 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:15.010947 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:15.011887 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:15.013000 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:15.014529 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:15.015542 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:15.015708 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:15.020254 (+ 4546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.1:60623: BlockingRecv error: recv got EOF from 127.2.120.1:60623 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 00:27:15 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:15 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:15.125316 13035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:15.114479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.3:48195)
0811 00:27:15.115633 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:15.115657 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:15.115878 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:15.116292 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:15.116297 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:15.116797 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:15.116979 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:15.117000 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:15.118309 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:15.118315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:15.119669 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:15.119689 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:15.120731 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:15.120736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:15.120867 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:15.121619 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:15.121640 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:15.124994 (+ 3354us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":952,"thread_start_us":514,"threads_started":1}
W20260811 00:27:15.125538 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:15.114950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:48195 (local address 127.2.120.62:37663)
0811 00:27:15.115338 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:15.115347 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:15.115791 (+ 444us) server_negotiation.cc:407] Connection header received
0811 00:27:15.116031 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:15.116037 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:15.116115 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:15.116231 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:15.117176 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:15.118071 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:15.119856 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:15.120500 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:15.120657 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:15.125264 (+ 4607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.3:48195: BlockingRecv error: recv got EOF from 127.2.120.3:48195 (error 108)
Metrics: {"server-negotiator.queue_time_us":133}
Aug 11 00:27:15 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:15 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:15.571568 13036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:15.557459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:54863)
0811 00:27:15.558430 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:15.558455 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:15.558674 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:15.559264 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:15.559268 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:15.559852 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:15.560070 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:15.560082 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:15.561871 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:15.561877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:15.563351 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:15.563390 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:15.564868 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:15.564878 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:15.565084 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:15.566045 (+ 961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:15.566069 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:15.571022 (+ 4953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":543,"threads_started":1}
W20260811 00:27:15.572754 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:15.557811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:54863 (local address 127.2.120.62:37663)
0811 00:27:15.558333 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:15.558345 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:15.558570 (+ 225us) server_negotiation.cc:407] Connection header received
0811 00:27:15.558879 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:15.558885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:15.559022 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:15.559160 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:15.560358 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:15.561533 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:15.563665 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:15.564437 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:15.564682 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:15.572286 (+ 7604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.2:54863: BlockingRecv error: recv got EOF from 127.2.120.2:54863 (error 108)
Metrics: {"server-negotiator.queue_time_us":227}
Aug 11 00:27:16 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:16 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:16.030510 13037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:16.021473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:51787)
0811 00:27:16.022046 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:16.022063 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:16.022238 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:16.022589 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:16.022593 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:16.023031 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:16.023198 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:16.023207 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:16.024214 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:16.024222 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:16.025239 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:16.025251 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:16.025991 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:16.025996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:16.026175 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:16.026817 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:16.026835 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:16.030106 (+ 3271us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":334,"threads_started":1}
W20260811 00:27:16.030625 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:16.021544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:51787 (local address 127.2.120.62:37663)
0811 00:27:16.021870 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:16.021879 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:16.022164 (+ 285us) server_negotiation.cc:407] Connection header received
0811 00:27:16.022364 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:16.022369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:16.022436 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:16.022544 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:16.023348 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:16.024069 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:16.025386 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:16.025822 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:16.025963 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:16.030382 (+ 4419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.1:51787: BlockingRecv error: recv got EOF from 127.2.120.1:51787 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 00:27:16 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:16 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:16.135852 13038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:16.126671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.3:39485)
0811 00:27:16.127301 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:16.127317 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:16.127450 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:16.127760 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:16.127763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:16.128201 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:16.128354 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:16.128363 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:16.129558 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:16.129562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:16.130495 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:16.130509 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:16.131309 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:16.131316 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:16.131468 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:16.132112 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:16.132131 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:16.135593 (+ 3462us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":354,"threads_started":1}
W20260811 00:27:16.136157 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:16.126811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:39485 (local address 127.2.120.62:37663)
0811 00:27:16.127026 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:16.127031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:16.127432 (+ 401us) server_negotiation.cc:407] Connection header received
0811 00:27:16.127562 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:16.127566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:16.127630 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:16.127720 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:16.128542 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:16.129382 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:16.130639 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:16.131091 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:16.131192 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:16.135824 (+ 4632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.3:39485: BlockingRecv error: recv got EOF from 127.2.120.3:39485 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 00:27:16 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:16 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:16.587456 13039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:16.574217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:40031)
0811 00:27:16.575595 (+ 1378us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:16.575634 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:16.576001 (+ 367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:16.577006 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:16.577016 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:16.577957 (+ 941us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:16.578344 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:16.578367 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:16.580225 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:16.580236 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:16.581506 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:16.581525 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:16.582549 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:16.582555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:16.582721 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:16.583499 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:16.583521 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:16.587146 (+ 3625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1178,"thread_start_us":912,"threads_started":1}
W20260811 00:27:16.587730 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:16.574577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:40031 (local address 127.2.120.62:37663)
0811 00:27:16.575195 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:16.575204 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:16.575856 (+ 652us) server_negotiation.cc:407] Connection header received
0811 00:27:16.576211 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:16.576237 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:16.576448 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:16.576657 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:16.578742 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:16.579935 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:16.581716 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:16.582260 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:16.582404 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:16.587446 (+ 5042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.2:40031: BlockingRecv error: recv got EOF from 127.2.120.2:40031 (error 108)
Metrics: {"server-negotiator.queue_time_us":271}
Aug 11 00:27:17 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:17 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:17.042259 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:17.031700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:44705)
0811 00:27:17.032296 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:17.032315 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:17.032441 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:17.032882 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:17.032886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:17.033323 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:17.033485 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:17.033494 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:17.034706 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:17.034710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:17.036010 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:17.036044 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:17.037416 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:17.037422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:17.037620 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:17.038393 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:17.038413 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:17.041905 (+ 3492us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":352,"threads_started":1}
W20260811 00:27:17.042481 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:17.031769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:44705 (local address 127.2.120.62:37663)
0811 00:27:17.031964 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:17.031969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:17.032429 (+ 460us) server_negotiation.cc:407] Connection header received
0811 00:27:17.032610 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:17.032615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:17.032696 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:17.032808 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:17.033684 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:17.034546 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:17.036340 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:17.037112 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:17.037404 (+ 292us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:17.042215 (+ 4811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.1:44705: BlockingRecv error: recv got EOF from 127.2.120.1:44705 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 00:27:17 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:17 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:17.156853 13042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:17.136960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.3:60259)
0811 00:27:17.145174 (+ 8214us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:17.145203 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:17.145407 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:17.146273 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:17.146279 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:17.146827 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:17.147162 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:17.147179 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:17.148682 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:17.148687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:17.149761 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:17.149777 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:17.151126 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:17.151134 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:17.151386 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:17.152359 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:17.152382 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:17.156470 (+ 4088us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":8043,"thread_start_us":7877,"threads_started":1}
W20260811 00:27:17.157152 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:17.137078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:60259 (local address 127.2.120.62:37663)
0811 00:27:17.137369 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:17.137378 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:17.145451 (+ 8073us) server_negotiation.cc:407] Connection header received
0811 00:27:17.145644 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:17.145651 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:17.145829 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:17.146145 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:17.147369 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:17.148484 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:17.150026 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:17.150838 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:17.150973 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:17.156852 (+ 5879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.3:60259: BlockingRecv error: recv got EOF from 127.2.120.3:60259 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 11 00:27:17 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:17 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:17.599828 13043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:17.589251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:43949)
0811 00:27:17.589982 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:17.590003 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:17.590228 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:17.590787 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:17.590793 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:17.591366 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:17.591568 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:17.591579 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:17.592937 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:17.592943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:17.594029 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:17.594044 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:17.594924 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:17.594931 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:17.595141 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:17.595889 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:17.595909 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:17.599537 (+ 3628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":394,"threads_started":1}
W20260811 00:27:17.600708 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:17.589366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:43949 (local address 127.2.120.62:37663)
0811 00:27:17.589622 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:17.589631 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:17.590095 (+ 464us) server_negotiation.cc:407] Connection header received
0811 00:27:17.590455 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:17.590461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:17.590560 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:17.590728 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:17.591772 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:17.592743 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:17.594230 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:17.594713 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:17.594872 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:17.600233 (+ 5361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.2:43949: BlockingRecv error: recv got EOF from 127.2.120.2:43949 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 00:27:18 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:18 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:18.059340 13044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:18.043634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:36541)
0811 00:27:18.044415 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:18.044434 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:18.044619 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:18.045049 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:18.045052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:18.045513 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:18.045715 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:18.045724 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:18.046938 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:18.046943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:18.048043 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:18.048060 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:18.049524 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:18.049531 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:18.049714 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:18.054105 (+ 4391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:18.054136 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:18.058941 (+ 4805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":501,"threads_started":1}
W20260811 00:27:18.059492 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:18.043801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:36541 (local address 127.2.120.62:37663)
0811 00:27:18.044135 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:18.044145 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:18.044598 (+ 453us) server_negotiation.cc:407] Connection header received
0811 00:27:18.044807 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:18.044815 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:18.044902 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:18.044991 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:18.045941 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:18.046756 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:18.048376 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:18.049130 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:18.049352 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:18.059190 (+ 9838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.1:36541: BlockingRecv error: recv got EOF from 127.2.120.1:36541 (error 108)
Metrics: {"server-negotiator.queue_time_us":158}
Aug 11 00:27:18 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:18 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:18.167590 13045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:18.158100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.3:36987)
0811 00:27:18.158673 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:18.158690 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:18.158862 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:18.159230 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:18.159235 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:18.159656 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:18.159800 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:18.159809 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:18.160921 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:18.160929 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:18.161926 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:18.161944 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:18.162863 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:18.162870 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:18.163067 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:18.163966 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:18.163986 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:18.167330 (+ 3344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":331,"threads_started":1}
W20260811 00:27:18.167954 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:18.158227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:36987 (local address 127.2.120.62:37663)
0811 00:27:18.158527 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:18.158535 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:18.158781 (+ 246us) server_negotiation.cc:407] Connection header received
0811 00:27:18.159017 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:18.159021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:18.159087 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:18.159198 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:18.159955 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:18.160788 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:18.162086 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:18.162687 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:18.162830 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:18.167634 (+ 4804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.3:36987: BlockingRecv error: recv got EOF from 127.2.120.3:36987 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 11 00:27:18 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:18 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:18.611584 13046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:18.601198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:43675)
0811 00:27:18.601835 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:18.601876 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:18.602049 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:18.602552 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:18.602557 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:18.603195 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:18.603454 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:18.603473 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:18.604631 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:18.604635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:18.605612 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:18.605626 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:18.606428 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:18.606432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:18.606577 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:18.607331 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:18.607350 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:18.610809 (+ 3459us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":442,"threads_started":1}
W20260811 00:27:18.611822 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:18.601441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:43675 (local address 127.2.120.62:37663)
0811 00:27:18.601654 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:18.601658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:18.601990 (+ 332us) server_negotiation.cc:407] Connection header received
0811 00:27:18.602219 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:18.602225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:18.602330 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:18.602453 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:18.603637 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:18.604421 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:18.605793 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:18.606256 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:18.606375 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:18.611412 (+ 5037us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.2:43675: BlockingRecv error: recv got EOF from 127.2.120.2:43675 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 00:27:19 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:19 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:19.070886 13047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:19.060597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:36195)
0811 00:27:19.061174 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:19.061189 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:19.061353 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:19.061746 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:19.061751 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:19.062248 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:19.062413 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:19.062422 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:19.063717 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:19.063723 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:19.064738 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:19.064752 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:19.065552 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:19.065559 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:19.065767 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:19.066500 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:19.066535 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:19.070516 (+ 3981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":348,"threads_started":1}
W20260811 00:27:19.071079 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:19.060691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:36195 (local address 127.2.120.62:37663)
0811 00:27:19.060992 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:19.060999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:19.061280 (+ 281us) server_negotiation.cc:407] Connection header received
0811 00:27:19.061506 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:19.061510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:19.061608 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:19.061713 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:19.062572 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:19.063539 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:19.064910 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:19.065359 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:19.065529 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:19.070790 (+ 5261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.1:36195: BlockingRecv error: recv got EOF from 127.2.120.1:36195 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
Aug 11 00:27:19 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:19 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:19.178853 13048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:19.168913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.3:36155)
0811 00:27:19.169487 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:19.169506 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:19.169692 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:19.170051 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:19.170055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:19.170528 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:19.170712 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:19.170722 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:19.171750 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:19.171755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:19.173292 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:19.173316 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:19.174201 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:19.174208 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:19.174420 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:19.175097 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:19.175116 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:19.178551 (+ 3435us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":317,"threads_started":1}
W20260811 00:27:19.179037 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:19.169047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:36155 (local address 127.2.120.62:37663)
0811 00:27:19.169310 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:19.169317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:19.169621 (+ 304us) server_negotiation.cc:407] Connection header received
0811 00:27:19.169828 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:19.169834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:19.169910 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:19.170017 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:19.170863 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:19.171594 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:19.173477 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:19.174012 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:19.174170 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:19.178816 (+ 4646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.3:36155: BlockingRecv error: recv got EOF from 127.2.120.3:36155 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 11 00:27:19 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:19 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:19.623086 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:19.612929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:54287)
0811 00:27:19.613671 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:19.613689 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:19.613875 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:19.614254 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:19.614261 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:19.614751 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:19.614926 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:19.614935 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:19.616082 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:19.616089 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:19.617233 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:19.617246 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:19.618090 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:19.618096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:19.618305 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:19.618998 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:19.619017 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:19.622681 (+ 3664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":382,"threads_started":1}
W20260811 00:27:19.623246 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:19.613186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:54287 (local address 127.2.120.62:37663)
0811 00:27:19.613490 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:19.613499 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:19.613793 (+ 294us) server_negotiation.cc:407] Connection header received
0811 00:27:19.614023 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:19.614028 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:19.614094 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:19.614208 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:19.615084 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:19.615925 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:19.617400 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:19.617915 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:19.618075 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:19.622967 (+ 4892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.2:54287: BlockingRecv error: recv got EOF from 127.2.120.2:54287 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 11 00:27:20 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:20 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:20.081395 13050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:20.072064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:43377)
0811 00:27:20.072672 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:20.072691 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:20.072829 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:20.073145 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:20.073148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:20.073602 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:20.073863 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:20.073875 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:20.075070 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:20.075084 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:20.076037 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:20.076049 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:20.077084 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:20.077089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:20.077224 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:20.077935 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:20.077955 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:20.081012 (+ 3057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":277,"threads_started":1}
W20260811 00:27:20.081727 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:20.072151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:43377 (local address 127.2.120.62:37663)
0811 00:27:20.072415 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:20.072422 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:20.072775 (+ 353us) server_negotiation.cc:407] Connection header received
0811 00:27:20.072934 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:20.072938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:20.073001 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:20.073086 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:20.074040 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:20.074860 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:20.076214 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:20.076723 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:20.077038 (+ 315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:20.081341 (+ 4303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.1:43377: BlockingRecv error: recv got EOF from 127.2.120.1:43377 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 11 00:27:20 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:20 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:20.189800 13051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:20.180009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.3:33851)
0811 00:27:20.180610 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:20.180629 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:20.180771 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:20.181119 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:20.181122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:20.181546 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:20.181706 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:20.181714 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:20.183182 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:20.183187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:20.184312 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:20.184335 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:20.185325 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:20.185332 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:20.185559 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:20.186426 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:20.186446 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:20.189543 (+ 3097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":322,"threads_started":1}
W20260811 00:27:20.190089 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:20.180122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:33851 (local address 127.2.120.62:37663)
0811 00:27:20.180363 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:20.180368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:20.180716 (+ 348us) server_negotiation.cc:407] Connection header received
0811 00:27:20.180903 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:20.180907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:20.180973 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:20.181070 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:20.181985 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:20.182944 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:20.184583 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:20.185107 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:20.185291 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:20.189743 (+ 4452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.3:33851: BlockingRecv error: recv got EOF from 127.2.120.3:33851 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 11 00:27:20 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:20 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:20.633750 13052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:20.624395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:51663)
0811 00:27:20.625102 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:20.625121 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:20.625286 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:20.625662 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:20.625667 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:20.626122 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:20.626289 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:20.626298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:20.627424 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:20.627432 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:20.628398 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:20.628412 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:20.629310 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:20.629316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:20.629520 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:20.630220 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:20.630239 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:20.633363 (+ 3124us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":404,"threads_started":1}
W20260811 00:27:20.633885 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:20.625105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:51663 (local address 127.2.120.62:37663)
0811 00:27:20.625339 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:20.625348 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:20.625368 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:27:20.625439 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:20.625445 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:20.625510 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:20.625628 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:20.626457 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:20.627259 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:20.628647 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:20.629100 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:20.629249 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:20.633625 (+ 4376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.2:51663: BlockingRecv error: recv got EOF from 127.2.120.2:51663 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 11 00:27:21 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:21 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:21.093715 13053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:21.082632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:34239)
0811 00:27:21.083233 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:21.083250 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:21.083387 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:21.083919 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:21.083923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:21.084327 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:21.084741 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:21.084763 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:21.086132 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:21.086137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:21.087356 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:21.087371 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:21.088992 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:21.088997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:21.089156 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:21.089936 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:21.089956 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:21.093313 (+ 3357us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":304,"threads_started":1}
W20260811 00:27:21.093802 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:21.082867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:34239 (local address 127.2.120.62:37663)
0811 00:27:21.083221 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:21.083227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:21.083442 (+ 215us) server_negotiation.cc:407] Connection header received
0811 00:27:21.083570 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:21.083575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:21.083702 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:21.083867 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:21.085003 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:21.085944 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:21.087599 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:21.088785 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:21.088946 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:21.093599 (+ 4653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.1:34239: BlockingRecv error: recv got EOF from 127.2.120.1:34239 (error 108)
Metrics: {"server-negotiator.queue_time_us":193}
Aug 11 00:27:21 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:21 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 00:27:21.200920 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:21.191041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:38011 (local address 127.2.120.62:37663)
0811 00:27:21.191363 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:21.191372 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:21.191744 (+ 372us) server_negotiation.cc:407] Connection header received
0811 00:27:21.191876 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:21.191881 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:21.191966 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:21.192094 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:21.192977 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:21.193720 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:21.194966 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:21.195422 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:21.195533 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:21.200701 (+ 5168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.3:38011: BlockingRecv error: recv got EOF from 127.2.120.3:38011 (error 108)
Metrics: {"server-negotiator.queue_time_us":161}
I20260811 00:27:21.200953 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:21.191013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.3:38011)
0811 00:27:21.191642 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:21.191660 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:21.191781 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:21.192184 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:21.192188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:21.192646 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:21.192811 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:21.192820 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:21.193864 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:21.193867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:21.194814 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:21.194826 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:21.195620 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:21.195635 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:21.195764 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:21.196495 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:21.196538 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:21.200346 (+ 3808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":338,"threads_started":1}
Aug 11 00:27:21 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:21 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:21.644922 13056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:21.635037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:50449)
0811 00:27:21.635977 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:21.636007 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:21.636170 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:21.636597 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:21.636602 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:21.637025 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:21.637219 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:21.637229 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:21.638375 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:21.638380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:21.639309 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:21.639322 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:21.640064 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:21.640068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:21.640194 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:21.640928 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:21.640949 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:21.644605 (+ 3656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":536,"threads_started":1}
W20260811 00:27:21.645151 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:21.635340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:50449 (local address 127.2.120.62:37663)
0811 00:27:21.635556 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:21.635562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:21.636114 (+ 552us) server_negotiation.cc:407] Connection header received
0811 00:27:21.636278 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:21.636282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:21.636348 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:21.636431 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:21.637396 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:21.638198 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:21.639485 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:21.639920 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:21.640028 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:21.644880 (+ 4852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.2:50449: BlockingRecv error: recv got EOF from 127.2.120.2:50449 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 00:27:22 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:22 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.1@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:22.104454 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:22.095239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.1:38027)
0811 00:27:22.096080 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:22.096096 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:22.096224 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:22.096609 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:22.096614 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:22.097108 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:22.097306 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:22.097315 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:22.098614 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:22.098618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:22.099619 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:22.099631 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:22.100274 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:22.100278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:22.100394 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:22.101018 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:22.101040 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:22.104191 (+ 3151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":727,"thread_start_us":629,"threads_started":1}
W20260811 00:27:22.104749 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:22.095461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:38027 (local address 127.2.120.62:37663)
0811 00:27:22.095711 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:22.095718 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:22.096207 (+ 489us) server_negotiation.cc:407] Connection header received
0811 00:27:22.096337 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:22.096340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:22.096405 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:22.096569 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:22.097445 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:22.098472 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:22.099749 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:22.100153 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:22.100263 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:22.104423 (+ 4160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.1:38027: BlockingRecv error: recv got EOF from 127.2.120.1:38027 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 11 00:27:22 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:22 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.3@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:22.212898 13058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:22.202270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.3:34159)
0811 00:27:22.202832 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:22.202850 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:22.202983 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:22.203397 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:22.203401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:22.203820 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:22.204030 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:22.204040 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:22.205513 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:22.205518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:22.206495 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:22.206509 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:22.207357 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:22.207363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:22.207550 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:22.208428 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:22.208451 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:22.212438 (+ 3987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":311,"threads_started":1}
W20260811 00:27:22.212960 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:22.202484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:34159 (local address 127.2.120.62:37663)
0811 00:27:22.202687 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:22.202691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:22.202973 (+ 282us) server_negotiation.cc:407] Connection header received
0811 00:27:22.203101 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:22.203105 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:22.203213 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:22.203317 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:22.204249 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:22.205253 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:22.206706 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:22.207137 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:22.207257 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:22.212691 (+ 5434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.3:34159: BlockingRecv error: recv got EOF from 127.2.120.3:34159 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 00:27:22 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:27:22 dist-test-slave-fd7l krb5kdc[11889](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.120.2@KRBTEST.COM for oryx/127.2.120.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:27:22.656790 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:22.646557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37663 (local address 127.2.120.2:49989)
0811 00:27:22.647219 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:22.647238 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:22.647386 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:22.647813 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:22.647816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:22.648298 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:22.648679 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:22.648691 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:22.649825 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:22.649830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:22.650816 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:22.650830 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:22.651668 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:22.651673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:22.651836 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:22.652551 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:22.652572 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:22.656417 (+ 3845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.120.62:37663: Server oryx/127.2.120.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":332,"threads_started":1}
W20260811 00:27:22.657035 12862 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:22.646857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:49989 (local address 127.2.120.62:37663)
0811 00:27:22.647101 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:22.647108 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:22.647386 (+ 278us) server_negotiation.cc:407] Connection header received
0811 00:27:22.647513 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:22.647526 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:22.647604 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:22.647690 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:22.648850 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:22.649645 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:22.650995 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:22.651478 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:22.651605 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:22.656760 (+ 5155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.120.2:49989: BlockingRecv error: recv got EOF from 127.2.120.2:49989 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 00:27:23.009406 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 12547
I20260811 00:27:23.025650 13061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:23.010962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49450 (local address 127.2.120.1:40441)
0811 00:27:23.011633 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:23.011641 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:23.011745 (+ 104us) server_negotiation.cc:407] Connection header received
0811 00:27:23.011942 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:23.011949 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:23.012197 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:23.012425 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:23.014151 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:23.015194 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:23.016659 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:23.017294 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:23.017407 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:23.020472 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:23.020488 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:23.020628 (+ 140us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:23.020710 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:23.024900 (+ 4190us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.120.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 00:27:23.025317 (+ 417us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49450: Request ticket server kudu/127.2.120.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":333,"threads_started":1}
W20260811 00:27:23.025831 13061 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49450: Request ticket server kudu/127.2.120.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 00:27:23.025942 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 12547: Not authorized: Client connection negotiation failed: client connection to 127.2.120.1:40441: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.120.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 00:27:23.026250 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 12547
I20260811 00:27:23.051712 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 12717
I20260811 00:27:23.070353 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:23.053063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51016 (local address 127.2.120.2:43241)
0811 00:27:23.053825 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:23.053833 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:23.053868 (+ 35us) server_negotiation.cc:407] Connection header received
0811 00:27:23.053936 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:23.053943 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:23.054283 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:23.054464 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:23.056098 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:23.057456 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:23.058834 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:23.059480 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:23.059600 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:23.066536 (+ 6936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:23.066550 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:23.066570 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:23.066636 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:23.069740 (+ 3104us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.120.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 00:27:23.070084 (+ 344us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51016: Request ticket server kudu/127.2.120.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":389,"threads_started":1}
W20260811 00:27:23.070457 13062 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51016: Request ticket server kudu/127.2.120.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 00:27:23.070720 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 12717: Not authorized: Client connection negotiation failed: client connection to 127.2.120.2:43241: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.120.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 00:27:23.070863 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 12717
I20260811 00:27:23.095810 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 12864
I20260811 00:27:23.111389 13063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:23.097011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49136 (local address 127.2.120.3:39301)
0811 00:27:23.097668 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:23.097676 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:23.097705 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:27:23.097773 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:23.097780 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:23.098034 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:23.098214 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:23.100030 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:23.101197 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:23.102544 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:23.103144 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:23.103304 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:23.107107 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:23.107120 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:23.107137 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:23.107186 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:23.110511 (+ 3325us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.120.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 00:27:23.110975 (+ 464us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49136: Request ticket server kudu/127.2.120.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":349,"threads_started":1}
W20260811 00:27:23.111552 13063 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49136: Request ticket server kudu/127.2.120.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 00:27:23.111611 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 12864: Not authorized: Client connection negotiation failed: client connection to 127.2.120.3:39301: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.120.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 00:27:23.111855 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 12864
I20260811 00:27:23.134902 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 12468
I20260811 00:27:23.136497 12540 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:27:23.260833 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 12468
2026-08-11T00:27:23Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22389 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T00:27:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:27:23Z Disabled control of system clock
I20260811 00:27:23.359273 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:33151
--webserver_interface=127.2.120.62
--webserver_port=0
--builtin_ntp_servers=127.2.120.20:36503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:33151
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 00:27:23.758154 13068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:23.758510 13068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:23.758653 13068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:23.766381 13068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:27:23.766534 13068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:23.766574 13068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:23.766709 13068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:27:23.766803 13068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:27:23.778306 13068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:36503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:33151
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:33151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:23.779992 13068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:23.782122 13068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:23.797297 13074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:23.797343 13077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:23.797288 13075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:23.798257 13068 server_base.cc:1034] running on GCE node
I20260811 00:27:23.799376 13068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:23.801512 13068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:23.802887 13068 hybrid_clock.cc:648] HybridClock initialized: now 1786408043802844 us; error 68 us; skew 500 ppm
I20260811 00:27:23.803365 13068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:27:23.806365 13068 webserver.cc:460] Webserver started at http://127.2.120.62:34021/ using document root <none> and password file <none>
I20260811 00:27:23.807272 13068 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:23.807456 13068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:23.807853 13068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:27:23.811702 13068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "ae88464460bd41f58ba434b6cf9a09db"
format_stamp: "Formatted at 2026-08-11 00:27:23 on dist-test-slave-fd7l"
server_key: "d922c002712a03951c3b3ebabc7bdea3"
server_key_iv: "8566f5eaf83d3957e8463056fc9b97eb"
server_key_version: "encryptionkey@0"
I20260811 00:27:23.812793 13068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "ae88464460bd41f58ba434b6cf9a09db"
format_stamp: "Formatted at 2026-08-11 00:27:23 on dist-test-slave-fd7l"
server_key: "d922c002712a03951c3b3ebabc7bdea3"
server_key_iv: "8566f5eaf83d3957e8463056fc9b97eb"
server_key_version: "encryptionkey@0"
I20260811 00:27:23.819199 13068 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:27:23.823653 13084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:23.825325 13068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:27:23.825558 13068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "ae88464460bd41f58ba434b6cf9a09db"
format_stamp: "Formatted at 2026-08-11 00:27:23 on dist-test-slave-fd7l"
server_key: "d922c002712a03951c3b3ebabc7bdea3"
server_key_iv: "8566f5eaf83d3957e8463056fc9b97eb"
server_key_version: "encryptionkey@0"
I20260811 00:27:23.825840 13068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:23.853223 13068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:23.854192 13068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:23.854529 13068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:23.874783 13068 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:33151
I20260811 00:27:23.874800 13146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:33151 every 8 connection(s)
I20260811 00:27:23.876344 13068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:27:23.880805 13147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:27:23.881567 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 13068
I20260811 00:27:23.881778 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:27:23.882489 2528 external_mini_cluster.cc:1428] Setting key f308ea285b0029bf361114909651f489
I20260811 00:27:23.897377 13147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db: Bootstrap starting.
I20260811 00:27:23.899936 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:23.884968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55166 (local address 127.2.120.62:33151)
0811 00:27:23.886259 (+ 1291us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:23.886278 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:23.886338 (+ 60us) server_negotiation.cc:407] Connection header received
0811 00:27:23.888310 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:23.888345 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:23.889102 (+ 757us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:23.889611 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:23.890204 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:23.892183 (+ 1979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:23.893794 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:23.895036 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:23.895243 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:23.896610 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:23.896624 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:23.896649 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:27:23.897442 (+ 793us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:23.897553 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:27:23.897590 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:23.897700 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:23.897967 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 00:27:23.898346 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":373,"threads_started":1}
I20260811 00:27:23.901961 13147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db: Neither blocks nor log segments found. Creating new log.
I20260811 00:27:23.903261 13147 log.cc:824] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db: Log is configured to *not* fsync() on all Append() calls
I20260811 00:27:23.906332 13147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db: No bootstrap required, opened a new log
I20260811 00:27:23.914904 13147 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae88464460bd41f58ba434b6cf9a09db" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33151 } }
I20260811 00:27:23.915369 13147 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:27:23.915477 13147 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae88464460bd41f58ba434b6cf9a09db, State: Initialized, Role: FOLLOWER
I20260811 00:27:23.916186 13147 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae88464460bd41f58ba434b6cf9a09db" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33151 } }
I20260811 00:27:23.916424 13147 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:27:23.916617 13147 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:27:23.916838 13147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:27:23.919211 13147 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae88464460bd41f58ba434b6cf9a09db" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33151 } }
I20260811 00:27:23.919778 13147 leader_election.cc:302] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae88464460bd41f58ba434b6cf9a09db; no voters:
I20260811 00:27:23.920431 13147 leader_election.cc:288] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:27:23.920684 13152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:27:23.921509 13152 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db [term 1 LEADER]: Becoming Leader. State: Replica: ae88464460bd41f58ba434b6cf9a09db, State: Running, Role: LEADER
I20260811 00:27:23.922171 13152 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae88464460bd41f58ba434b6cf9a09db" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33151 } }
I20260811 00:27:23.922865 13147 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:27:23.925339 13153 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae88464460bd41f58ba434b6cf9a09db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae88464460bd41f58ba434b6cf9a09db" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33151 } } }
I20260811 00:27:23.925489 13154 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae88464460bd41f58ba434b6cf9a09db. Latest consensus state: current_term: 1 leader_uuid: "ae88464460bd41f58ba434b6cf9a09db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae88464460bd41f58ba434b6cf9a09db" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 33151 } } }
I20260811 00:27:23.926193 13154 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db [sys.catalog]: This master's current role is: LEADER
I20260811 00:27:23.926052 13153 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db [sys.catalog]: This master's current role is: LEADER
I20260811 00:27:23.928290 13158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:27:23.937729 13158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:27:23.948118 13158 catalog_manager.cc:1347] Generated new cluster ID: 475930f084a34efcbcba467d95a6e5f9
I20260811 00:27:23.948282 13158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:27:23.960040 13158 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:27:23.961733 13158 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:27:23.970326 13158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae88464460bd41f58ba434b6cf9a09db: Generated new TSK 0
I20260811 00:27:23.971549 13158 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:27:23.991802 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:33151
--builtin_ntp_servers=127.2.120.20:36503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 00:27:24.389240 13172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:24.389590 13172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:24.389683 13172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:24.399016 13172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:24.399149 13172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:24.399262 13172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:27:24.409444 13172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:36503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:33151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:24.410907 13172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:24.412940 13172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:24.428758 13177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:24.428900 13180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:24.428716 13178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:24.429704 13172 server_base.cc:1034] running on GCE node
I20260811 00:27:24.430595 13172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:24.431692 13172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:24.432947 13172 hybrid_clock.cc:648] HybridClock initialized: now 1786408044432870 us; error 76 us; skew 500 ppm
I20260811 00:27:24.433326 13172 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:27:24.436126 13172 webserver.cc:460] Webserver started at http://127.2.120.1:34965/ using document root <none> and password file <none>
I20260811 00:27:24.437017 13172 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:24.437179 13172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:24.437552 13172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:27:24.440937 13172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "8b6f44924dc84a8d9a532844ad2d7150"
format_stamp: "Formatted at 2026-08-11 00:27:24 on dist-test-slave-fd7l"
server_key: "ef31d075eb73813f8100cddc511905bd"
server_key_iv: "eee7c2412c6d038a7f5e661e425eb6fd"
server_key_version: "encryptionkey@0"
I20260811 00:27:24.441859 13172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "8b6f44924dc84a8d9a532844ad2d7150"
format_stamp: "Formatted at 2026-08-11 00:27:24 on dist-test-slave-fd7l"
server_key: "ef31d075eb73813f8100cddc511905bd"
server_key_iv: "eee7c2412c6d038a7f5e661e425eb6fd"
server_key_version: "encryptionkey@0"
I20260811 00:27:24.447695 13172 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:27:24.452318 13186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:24.454010 13172 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:27:24.454226 13172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "8b6f44924dc84a8d9a532844ad2d7150"
format_stamp: "Formatted at 2026-08-11 00:27:24 on dist-test-slave-fd7l"
server_key: "ef31d075eb73813f8100cddc511905bd"
server_key_iv: "eee7c2412c6d038a7f5e661e425eb6fd"
server_key_version: "encryptionkey@0"
I20260811 00:27:24.454478 13172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:24.476164 13172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:24.477290 13172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:24.477622 13172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:24.478731 13172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:27:24.480293 13172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:27:24.480428 13172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:24.480609 13172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:27:24.480718 13172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:24.521203 13172 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:39949
I20260811 00:27:24.521311 13316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:39949 every 8 connection(s)
I20260811 00:27:24.522789 13172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:27:24.529774 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 13172
I20260811 00:27:24.530009 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:27:24.530841 2528 external_mini_cluster.cc:1428] Setting key c51bfa5fc159ab15ab2ae7f67b332f97
I20260811 00:27:24.533980 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:33151
--builtin_ntp_servers=127.2.120.20:36503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 00:27:24.538884 13320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:24.525839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:55627 (local address 127.2.120.62:33151)
0811 00:27:24.526337 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:24.526344 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:24.527918 (+ 1574us) server_negotiation.cc:407] Connection header received
0811 00:27:24.530402 (+ 2484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:24.530411 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:24.530521 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:24.530650 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:24.532146 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:24.533205 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:24.535067 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:24.535867 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:24.536049 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:24.537746 (+ 1697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:24.537756 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:24.537763 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:27:24.537793 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:24.537847 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:27:24.537866 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:24.537979 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:24.538307 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 00:27:24.538432 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":232,"threads_started":1}
I20260811 00:27:24.539950 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:24.526486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33151 (local address 127.2.120.1:55627)
0811 00:27:24.527686 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:24.527735 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:24.529862 (+ 2127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:24.530955 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:24.530969 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:24.531098 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:24.531912 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:24.531937 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:24.533388 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:24.533392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:24.534873 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:24.534890 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:24.536158 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:24.536164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:24.536421 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:24.537329 (+ 908us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:27:24.537332 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:24.537378 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:27:24.537482 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:27:24.538025 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:27:24.538046 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:27:24.538054 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 00:27:24.538270 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 00:27:24.538657 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":325,"threads_started":1}
I20260811 00:27:24.541962 13317 heartbeater.cc:347] Connected to a master server at 127.2.120.62:33151
I20260811 00:27:24.542352 13317 heartbeater.cc:464] Registering TS with master...
I20260811 00:27:24.543278 13317 heartbeater.cc:511] Master 127.2.120.62:33151 requested a full tablet report, sending...
I20260811 00:27:24.546229 13105 ts_manager.cc:194] Registered new tserver with Master: 8b6f44924dc84a8d9a532844ad2d7150 (127.2.120.1:39949)
I20260811 00:27:24.547971 13105 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.120.1:55627
W20260811 00:27:24.923488 13322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:24.923877 13322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:24.924034 13322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:24.932039 13322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:24.932199 13322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:24.932376 13322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:27:24.941999 13322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:36503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:33151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:24.943497 13322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:24.945627 13322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:24.960920 13329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:24.960865 13332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:24.960893 13330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:24.962021 13322 server_base.cc:1034] running on GCE node
I20260811 00:27:24.962836 13322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:24.964046 13322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:24.967549 13322 hybrid_clock.cc:648] HybridClock initialized: now 1786408044967385 us; error 151 us; skew 500 ppm
I20260811 00:27:24.968006 13322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:27:24.971033 13322 webserver.cc:460] Webserver started at http://127.2.120.2:34421/ using document root <none> and password file <none>
I20260811 00:27:24.971892 13322 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:24.972059 13322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:24.972443 13322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:27:24.975956 13322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "3eabc88181a543c899df550d199bfc8c"
format_stamp: "Formatted at 2026-08-11 00:27:24 on dist-test-slave-fd7l"
server_key: "0d26a59018f29a94d431da25391db2f5"
server_key_iv: "d9632bb533f88e0f69681475b465af3c"
server_key_version: "encryptionkey@0"
I20260811 00:27:24.977041 13322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "3eabc88181a543c899df550d199bfc8c"
format_stamp: "Formatted at 2026-08-11 00:27:24 on dist-test-slave-fd7l"
server_key: "0d26a59018f29a94d431da25391db2f5"
server_key_iv: "d9632bb533f88e0f69681475b465af3c"
server_key_version: "encryptionkey@0"
I20260811 00:27:24.984084 13322 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260811 00:27:24.988840 13338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:24.990779 13322 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260811 00:27:24.991047 13322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "3eabc88181a543c899df550d199bfc8c"
format_stamp: "Formatted at 2026-08-11 00:27:24 on dist-test-slave-fd7l"
server_key: "0d26a59018f29a94d431da25391db2f5"
server_key_iv: "d9632bb533f88e0f69681475b465af3c"
server_key_version: "encryptionkey@0"
I20260811 00:27:24.991345 13322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:25.006734 13322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:25.007735 13322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:25.008045 13322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:25.009311 13322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:27:25.011015 13322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:27:25.011170 13322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:25.011309 13322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:27:25.011400 13322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:25.049163 13322 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:45985
I20260811 00:27:25.049261 13468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:45985 every 8 connection(s)
I20260811 00:27:25.050597 13322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:27:25.057575 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 13322
I20260811 00:27:25.057777 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:27:25.058292 2528 external_mini_cluster.cc:1428] Setting key 270c8fba32d8b0befe1bf00f133798df
I20260811 00:27:25.061260 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:33151
--builtin_ntp_servers=127.2.120.20:36503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 00:27:25.065294 13472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:25.053427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:58951 (local address 127.2.120.62:33151)
0811 00:27:25.053960 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:25.053965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:25.055130 (+ 1165us) server_negotiation.cc:407] Connection header received
0811 00:27:25.057181 (+ 2051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:25.057187 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:25.057290 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:25.057422 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:25.058832 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:25.059864 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:25.062008 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:25.062494 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:25.062679 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:25.064250 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:25.064256 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:25.064259 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:27:25.064280 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:25.064323 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:27:25.064336 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:25.064445 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:25.064770 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 00:27:25.064887 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":303,"threads_started":1}
I20260811 00:27:25.066457 13473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:25.053913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33151 (local address 127.2.120.2:58951)
0811 00:27:25.054947 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:25.054984 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:25.056860 (+ 1876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:25.057673 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:25.057683 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:25.057822 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:25.058645 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:25.058667 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:25.060042 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:25.060053 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:25.061840 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:25.061856 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:25.062780 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:25.062788 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:25.063048 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:25.063872 (+ 824us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:27:25.063875 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:25.063920 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:27:25.064014 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:27:25.064447 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:27:25.064462 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:27:25.064474 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 00:27:25.064759 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 00:27:25.065234 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":304,"threads_started":1}
I20260811 00:27:25.068130 13469 heartbeater.cc:347] Connected to a master server at 127.2.120.62:33151
I20260811 00:27:25.068482 13469 heartbeater.cc:464] Registering TS with master...
I20260811 00:27:25.069407 13469 heartbeater.cc:511] Master 127.2.120.62:33151 requested a full tablet report, sending...
I20260811 00:27:25.071070 13105 ts_manager.cc:194] Registered new tserver with Master: 3eabc88181a543c899df550d199bfc8c (127.2.120.2:45985)
I20260811 00:27:25.071923 13105 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.120.2:58951
W20260811 00:27:25.431118 13474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:25.431473 13474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:25.431627 13474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:25.439108 13474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:25.439224 13474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:25.439393 13474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:27:25.448927 13474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:36503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:33151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:25.450367 13474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:25.452270 13474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:25.471180 13482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:25.472098 13480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:25.474417 13479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:25.476452 13474 server_base.cc:1034] running on GCE node
I20260811 00:27:25.477279 13474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:25.478422 13474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:25.479719 13474 hybrid_clock.cc:648] HybridClock initialized: now 1786408045479637 us; error 73 us; skew 500 ppm
I20260811 00:27:25.480157 13474 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:27:25.482873 13474 webserver.cc:460] Webserver started at http://127.2.120.3:38587/ using document root <none> and password file <none>
I20260811 00:27:25.483724 13474 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:25.483879 13474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:25.484251 13474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:27:25.490520 13474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "ae469d2acfd342e19deb810d06bdab8a"
format_stamp: "Formatted at 2026-08-11 00:27:25 on dist-test-slave-fd7l"
server_key: "50b2c7ce007de64a5f3f493c9a3e50a5"
server_key_iv: "8d01bc4c1df16cd1de9b3128c48b8654"
server_key_version: "encryptionkey@0"
I20260811 00:27:25.491452 13474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "ae469d2acfd342e19deb810d06bdab8a"
format_stamp: "Formatted at 2026-08-11 00:27:25 on dist-test-slave-fd7l"
server_key: "50b2c7ce007de64a5f3f493c9a3e50a5"
server_key_iv: "8d01bc4c1df16cd1de9b3128c48b8654"
server_key_version: "encryptionkey@0"
I20260811 00:27:25.497366 13474 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 00:27:25.501358 13489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:25.502890 13474 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 00:27:25.503098 13474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "ae469d2acfd342e19deb810d06bdab8a"
format_stamp: "Formatted at 2026-08-11 00:27:25 on dist-test-slave-fd7l"
server_key: "50b2c7ce007de64a5f3f493c9a3e50a5"
server_key_iv: "8d01bc4c1df16cd1de9b3128c48b8654"
server_key_version: "encryptionkey@0"
I20260811 00:27:25.503368 13474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:25.524065 13474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:25.525065 13474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:25.525380 13474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:25.526408 13474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:27:25.527927 13474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:27:25.528057 13474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:25.528201 13474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:27:25.528293 13474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:25.551959 13317 heartbeater.cc:503] Master 127.2.120.62:33151 was elected leader, sending a full tablet report...
I20260811 00:27:25.567126 13474 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:36749
I20260811 00:27:25.567145 13617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:36749 every 8 connection(s)
I20260811 00:27:25.568668 13474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:27:25.572248 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 13474
I20260811 00:27:25.572451 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:27:25.572978 2528 external_mini_cluster.cc:1428] Setting key 7a98ede42a57cc6075156316b0147a8f
I20260811 00:27:25.583259 13623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:25.571597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:34073 (local address 127.2.120.62:33151)
0811 00:27:25.572232 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:25.572241 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:25.575751 (+ 3510us) server_negotiation.cc:407] Connection header received
0811 00:27:25.575812 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:25.575816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:25.575885 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:25.576025 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:25.577487 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:25.578298 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:25.580089 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:25.580567 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:25.580725 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:25.582287 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:25.582293 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:25.582296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:27:25.582318 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:25.582363 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:27:25.582376 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:25.582482 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:25.582739 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 00:27:25.582853 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":290,"threads_started":1}
I20260811 00:27:25.584393 13624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:25.572260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:33151 (local address 127.2.120.3:34073)
0811 00:27:25.573600 (+ 1340us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:25.573650 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:25.575395 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:25.576365 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:25.576380 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:25.576560 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:25.577269 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:25.577290 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:25.578492 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:25.578503 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:25.579877 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:25.579895 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:25.580836 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:25.580843 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:25.581111 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:25.581888 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:27:25.581891 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:25.581943 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:27:25.582042 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:27:25.582485 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:27:25.582500 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:27:25.582509 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 00:27:25.582728 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 00:27:25.583198 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"thread_start_us":477,"threads_started":1}
I20260811 00:27:25.586241 13619 heartbeater.cc:347] Connected to a master server at 127.2.120.62:33151
I20260811 00:27:25.586676 13619 heartbeater.cc:464] Registering TS with master...
I20260811 00:27:25.587602 13619 heartbeater.cc:511] Master 127.2.120.62:33151 requested a full tablet report, sending...
I20260811 00:27:25.589443 13104 ts_manager.cc:194] Registered new tserver with Master: ae469d2acfd342e19deb810d06bdab8a (127.2.120.3:36749)
I20260811 00:27:25.590446 13104 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.120.3:34073
I20260811 00:27:25.603415 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 00:27:25.611521 13623 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:25.609772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55178 (local address 127.2.120.62:33151)
0811 00:27:25.610069 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:25.610080 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:25.610492 (+ 412us) server_negotiation.cc:407] Connection header received
0811 00:27:25.610647 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:25.610651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:25.610723 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:25.610815 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:25.611256 (+ 441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55178: BlockingRecv error: recv got EOF from 127.0.0.1:55178 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 00:27:25.612455 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 13172
I20260811 00:27:25.621407 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:25.613627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54254 (local address 127.2.120.1:39949)
0811 00:27:25.614226 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:25.614234 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:25.614261 (+ 27us) server_negotiation.cc:407] Connection header received
0811 00:27:25.614328 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:25.614335 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:25.614575 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:25.614810 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:25.615265 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:25.616279 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:25.617689 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:25.618307 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:25.618474 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:25.619713 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:25.619722 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:25.619738 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:27:25.620406 (+ 668us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:25.620470 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:27:25.620501 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:25.620695 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:25.620866 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 00:27:25.621029 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":281,"threads_started":1}
I20260811 00:27:25.622900 13312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:27:25.707206 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 13172
I20260811 00:27:25.730378 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 13322
I20260811 00:27:25.739300 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:25.731516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43636 (local address 127.2.120.2:45985)
0811 00:27:25.732028 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:25.732036 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:25.732065 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:27:25.732141 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:25.732147 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:25.732374 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:25.732603 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:25.733041 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:25.734035 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:25.735346 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:25.735957 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:25.736151 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:25.737650 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:25.737661 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:25.737676 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:27:25.738358 (+ 682us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:25.738413 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:27:25.738435 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:25.738585 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:25.738766 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 00:27:25.738918 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":250,"threads_started":1}
I20260811 00:27:25.740793 13463 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:27:25.828576 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 13322
I20260811 00:27:25.850355 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 13474
I20260811 00:27:25.859095 13635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:25.851438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40700 (local address 127.2.120.3:36749)
0811 00:27:25.852110 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:25.852117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:25.852149 (+ 32us) server_negotiation.cc:407] Connection header received
0811 00:27:25.852209 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:25.852215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:25.852451 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:25.852666 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:25.853116 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:25.854179 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:25.855422 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:25.856011 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:25.856166 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:25.857495 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:25.857505 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:25.857520 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:27:25.858151 (+ 631us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:25.858206 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:27:25.858228 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:25.858366 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:25.858538 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 00:27:25.858684 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":363,"threads_started":1}
I20260811 00:27:25.860414 13613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:27:25.952224 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 13474
I20260811 00:27:25.972909 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 13068
I20260811 00:27:25.974412 13142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:27:26.087524 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 13068
2026-08-11T00:27:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2820 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T00:27:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:27:26Z Disabled control of system clock
I20260811 00:27:26.178977 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:34851
--webserver_interface=127.2.120.62
--webserver_port=0
--builtin_ntp_servers=127.2.120.20:36097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:34851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 00:27:26.568892 13641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:26.569239 13641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:26.569329 13641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:26.577028 13641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:27:26.577167 13641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:26.577206 13641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:26.577257 13641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:27:26.577296 13641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:27:26.587639 13641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:36097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:34851
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:34851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:26.589219 13641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:26.591187 13641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:26.607438 13646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:26.607373 13647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:26.607415 13650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:26.610493 13641 server_base.cc:1034] running on GCE node
I20260811 00:27:26.611547 13641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:26.614651 13641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:26.616058 13641 hybrid_clock.cc:648] HybridClock initialized: now 1786408046616001 us; error 45 us; skew 500 ppm
I20260811 00:27:26.616791 13641 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:27:26.620580 13641 webserver.cc:460] Webserver started at http://127.2.120.62:40973/ using document root <none> and password file <none>
I20260811 00:27:26.621475 13641 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:26.621637 13641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:26.622058 13641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:27:26.625790 13641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "0558c4bf54d543c28a68c7bd1df14e91"
format_stamp: "Formatted at 2026-08-11 00:27:26 on dist-test-slave-fd7l"
server_key: "4ca94f44523e53188cb5fabd21bddd00"
server_key_iv: "a7e4c3e9c2a0e1a54559b24d3c799e60"
server_key_version: "encryptionkey@0"
I20260811 00:27:26.626758 13641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "0558c4bf54d543c28a68c7bd1df14e91"
format_stamp: "Formatted at 2026-08-11 00:27:26 on dist-test-slave-fd7l"
server_key: "4ca94f44523e53188cb5fabd21bddd00"
server_key_iv: "a7e4c3e9c2a0e1a54559b24d3c799e60"
server_key_version: "encryptionkey@0"
I20260811 00:27:26.633684 13641 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 00:27:26.639945 13657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:26.642100 13641 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 00:27:26.642410 13641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "0558c4bf54d543c28a68c7bd1df14e91"
format_stamp: "Formatted at 2026-08-11 00:27:26 on dist-test-slave-fd7l"
server_key: "4ca94f44523e53188cb5fabd21bddd00"
server_key_iv: "a7e4c3e9c2a0e1a54559b24d3c799e60"
server_key_version: "encryptionkey@0"
I20260811 00:27:26.642747 13641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:26.649842 13641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:26.650818 13641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:26.651166 13641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:26.674793 13641 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:34851
I20260811 00:27:26.674813 13712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:34851 every 8 connection(s)
I20260811 00:27:26.677605 13641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:27:26.681134 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 13641
I20260811 00:27:26.681387 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:27:26.682292 2528 external_mini_cluster.cc:1428] Setting key 6683656e78147932a69fd0970b97f72a
I20260811 00:27:26.683281 13713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:27:26.692498 13716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:26.685318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45406 (local address 127.2.120.62:34851)
0811 00:27:26.686515 (+ 1197us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:26.686526 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:26.686576 (+ 50us) server_negotiation.cc:407] Connection header received
0811 00:27:26.688434 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:26.688460 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:26.689037 (+ 577us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:26.689437 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:26.690726 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:26.690733 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:26.690750 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:27:26.690769 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:26.690842 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:27:26.690860 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:26.691066 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:26.691174 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 00:27:26.691525 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":466,"threads_started":1}
I20260811 00:27:26.698599 13713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91: Bootstrap starting.
I20260811 00:27:26.702627 13713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91: Neither blocks nor log segments found. Creating new log.
I20260811 00:27:26.704219 13713 log.cc:824] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91: Log is configured to *not* fsync() on all Append() calls
I20260811 00:27:26.708860 13713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91: No bootstrap required, opened a new log
I20260811 00:27:26.717628 13713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0558c4bf54d543c28a68c7bd1df14e91" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 34851 } }
I20260811 00:27:26.718266 13713 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:27:26.718407 13713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0558c4bf54d543c28a68c7bd1df14e91, State: Initialized, Role: FOLLOWER
I20260811 00:27:26.719255 13713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0558c4bf54d543c28a68c7bd1df14e91" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 34851 } }
I20260811 00:27:26.719575 13713 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:27:26.719738 13713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:27:26.719925 13713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:27:26.722748 13713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0558c4bf54d543c28a68c7bd1df14e91" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 34851 } }
I20260811 00:27:26.723639 13713 leader_election.cc:302] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0558c4bf54d543c28a68c7bd1df14e91; no voters:
I20260811 00:27:26.724673 13713 leader_election.cc:288] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:27:26.724862 13718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:27:26.726202 13718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91 [term 1 LEADER]: Becoming Leader. State: Replica: 0558c4bf54d543c28a68c7bd1df14e91, State: Running, Role: LEADER
I20260811 00:27:26.726823 13718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0558c4bf54d543c28a68c7bd1df14e91" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 34851 } }
I20260811 00:27:26.727289 13713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:27:26.729918 13719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0558c4bf54d543c28a68c7bd1df14e91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0558c4bf54d543c28a68c7bd1df14e91" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 34851 } } }
I20260811 00:27:26.729911 13720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0558c4bf54d543c28a68c7bd1df14e91. Latest consensus state: current_term: 1 leader_uuid: "0558c4bf54d543c28a68c7bd1df14e91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0558c4bf54d543c28a68c7bd1df14e91" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 34851 } } }
I20260811 00:27:26.730432 13719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91 [sys.catalog]: This master's current role is: LEADER
I20260811 00:27:26.730535 13720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91 [sys.catalog]: This master's current role is: LEADER
I20260811 00:27:26.732986 13726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:27:26.738193 13726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:27:26.748693 13726 catalog_manager.cc:1347] Generated new cluster ID: 87dfa1fbc1344eec857baf54dae98ac8
I20260811 00:27:26.748838 13726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:27:26.768939 13726 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:27:26.769498 13726 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:27:26.779335 13726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0558c4bf54d543c28a68c7bd1df14e91: Generated new TSK 0
I20260811 00:27:26.780866 13726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:27:26.789343 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:34851
--builtin_ntp_servers=127.2.120.20:36097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 00:27:27.209616 13740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:27.209949 13740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:27.210119 13740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:27.217700 13740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:27.217831 13740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:27.218009 13740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:27:27.228039 13740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:36097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:34851
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:27.229605 13740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:27.231642 13740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:27.246656 13746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:27.246656 13747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:27.247064 13749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:27.247743 13740 server_base.cc:1034] running on GCE node
I20260811 00:27:27.248557 13740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:27.249637 13740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:27.250902 13740 hybrid_clock.cc:648] HybridClock initialized: now 1786408047250833 us; error 67 us; skew 500 ppm
I20260811 00:27:27.251307 13740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:27:27.253803 13740 webserver.cc:460] Webserver started at http://127.2.120.1:45277/ using document root <none> and password file <none>
I20260811 00:27:27.254657 13740 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:27.254752 13740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:27.255046 13740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:27:27.258186 13740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "fd30c03e170d4ee3a345310afae67656"
format_stamp: "Formatted at 2026-08-11 00:27:27 on dist-test-slave-fd7l"
server_key: "79a0b4bbd5f8f11b344400d4cb77eb1a"
server_key_iv: "00d8c50d4ce1104a59b310eea9b0eb24"
server_key_version: "encryptionkey@0"
I20260811 00:27:27.259033 13740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "fd30c03e170d4ee3a345310afae67656"
format_stamp: "Formatted at 2026-08-11 00:27:27 on dist-test-slave-fd7l"
server_key: "79a0b4bbd5f8f11b344400d4cb77eb1a"
server_key_iv: "00d8c50d4ce1104a59b310eea9b0eb24"
server_key_version: "encryptionkey@0"
I20260811 00:27:27.265101 13740 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:27:27.269321 13756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:27.271257 13740 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:27:27.271472 13740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "fd30c03e170d4ee3a345310afae67656"
format_stamp: "Formatted at 2026-08-11 00:27:27 on dist-test-slave-fd7l"
server_key: "79a0b4bbd5f8f11b344400d4cb77eb1a"
server_key_iv: "00d8c50d4ce1104a59b310eea9b0eb24"
server_key_version: "encryptionkey@0"
I20260811 00:27:27.271734 13740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:27.278442 13740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:27.279464 13740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:27.279781 13740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:27.280925 13740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:27:27.282545 13740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:27:27.282675 13740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:27.282809 13740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:27:27.282895 13740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:27.318758 13740 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:40753
I20260811 00:27:27.318858 13885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:40753 every 8 connection(s)
I20260811 00:27:27.320180 13740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:27:27.323463 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 13740
I20260811 00:27:27.323719 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:27:27.324378 2528 external_mini_cluster.cc:1428] Setting key 538a9e91ffd2db311e6e2afee15dc130
I20260811 00:27:27.328240 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:34851
--builtin_ntp_servers=127.2.120.20:36097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 00:27:27.330622 13889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:27.323092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:47809 (local address 127.2.120.62:34851)
0811 00:27:27.323628 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:27.323635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:27.325191 (+ 1556us) server_negotiation.cc:407] Connection header received
0811 00:27:27.327830 (+ 2639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:27.327839 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:27.327910 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:27.328044 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:27.329664 (+ 1620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:27.329672 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:27.329680 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:27:27.329705 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:27.329768 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:27:27.329789 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:27.329930 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:27.330131 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 00:27:27.330202 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":256,"threads_started":1}
I20260811 00:27:27.331482 13890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:27.323749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:34851 (local address 127.2.120.1:47809)
0811 00:27:27.324986 (+ 1237us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:27.325035 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:27.327516 (+ 2481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:27.328343 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:27.328353 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:27.328506 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:27.329321 (+ 815us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:27:27.329324 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:27.329375 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:27:27.329468 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:27:27.329903 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:27:27.329912 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:27:27.329918 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 00:27:27.330182 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 00:27:27.330410 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":372,"threads_started":1}
I20260811 00:27:27.333235 13887 heartbeater.cc:347] Connected to a master server at 127.2.120.62:34851
I20260811 00:27:27.333559 13887 heartbeater.cc:464] Registering TS with master...
I20260811 00:27:27.334182 13887 heartbeater.cc:511] Master 127.2.120.62:34851 requested a full tablet report, sending...
I20260811 00:27:27.336277 13677 ts_manager.cc:194] Registered new tserver with Master: fd30c03e170d4ee3a345310afae67656 (127.2.120.1:40753)
W20260811 00:27:27.713414 13892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:27.713783 13892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:27.713973 13892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:27.722604 13892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:27.722806 13892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:27.723012 13892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:27:27.733913 13892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:36097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:34851
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:27.735496 13892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:27.737874 13892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:27.754307 13898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:27.754413 13899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:27.754457 13901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:27.755833 13892 server_base.cc:1034] running on GCE node
I20260811 00:27:27.756680 13892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:27.757776 13892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:27.759050 13892 hybrid_clock.cc:648] HybridClock initialized: now 1786408047758976 us; error 70 us; skew 500 ppm
I20260811 00:27:27.759423 13892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:27:27.762176 13892 webserver.cc:460] Webserver started at http://127.2.120.2:35371/ using document root <none> and password file <none>
I20260811 00:27:27.762943 13892 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:27.763083 13892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:27.763516 13892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:27:27.767024 13892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "77d592539883456abd00cb78bf7a2e31"
format_stamp: "Formatted at 2026-08-11 00:27:27 on dist-test-slave-fd7l"
server_key: "72ae7e1c3943eeb679e446f6e457244a"
server_key_iv: "9fbb979bdf2a678a2addcea8c0c5a811"
server_key_version: "encryptionkey@0"
I20260811 00:27:27.767922 13892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "77d592539883456abd00cb78bf7a2e31"
format_stamp: "Formatted at 2026-08-11 00:27:27 on dist-test-slave-fd7l"
server_key: "72ae7e1c3943eeb679e446f6e457244a"
server_key_iv: "9fbb979bdf2a678a2addcea8c0c5a811"
server_key_version: "encryptionkey@0"
I20260811 00:27:27.774096 13892 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 00:27:27.778729 13907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:27.780691 13892 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:27:27.780913 13892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "77d592539883456abd00cb78bf7a2e31"
format_stamp: "Formatted at 2026-08-11 00:27:27 on dist-test-slave-fd7l"
server_key: "72ae7e1c3943eeb679e446f6e457244a"
server_key_iv: "9fbb979bdf2a678a2addcea8c0c5a811"
server_key_version: "encryptionkey@0"
I20260811 00:27:27.781199 13892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:27.788101 13892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:27.789258 13892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:27.789618 13892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:27.790781 13892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:27:27.792606 13892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:27:27.792764 13892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 00:27:27.792923 13892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:27:27.793035 13892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:27.832408 13892 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:35527
I20260811 00:27:27.832409 14033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:35527 every 8 connection(s)
I20260811 00:27:27.834142 13892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:27:27.840065 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 13892
I20260811 00:27:27.840301 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:27:27.840914 2528 external_mini_cluster.cc:1428] Setting key 588454361369c49c53ce6cdcce7d0e60
I20260811 00:27:27.844095 14036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:27.837198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:50939 (local address 127.2.120.62:34851)
0811 00:27:27.837680 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:27.837685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:27.838966 (+ 1281us) server_negotiation.cc:407] Connection header received
0811 00:27:27.840974 (+ 2008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:27.840980 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:27.841064 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:27.841216 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:27.843273 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:27.843279 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:27.843284 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:27:27.843305 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:27.843361 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:27:27.843374 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:27.843516 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:27.843730 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 00:27:27.843803 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":240,"threads_started":1}
I20260811 00:27:27.844457 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:34851
--builtin_ntp_servers=127.2.120.20:36097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 00:27:27.845329 14037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:27.837832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:34851 (local address 127.2.120.2:50939)
0811 00:27:27.838787 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:27.838825 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:27.840635 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:27.841727 (+ 1092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:27.841742 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:27.841920 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:27.842799 (+ 879us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:27:27.842802 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:27.842845 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:27:27.843104 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:27:27.843477 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:27:27.843491 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:27:27.843501 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 00:27:27.843830 (+ 329us) client_negotiation.cc:240] Negotiation successful
0811 00:27:27.844155 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":288,"threads_started":1}
I20260811 00:27:27.846928 14034 heartbeater.cc:347] Connected to a master server at 127.2.120.62:34851
I20260811 00:27:27.847213 14034 heartbeater.cc:464] Registering TS with master...
I20260811 00:27:27.847947 14034 heartbeater.cc:511] Master 127.2.120.62:34851 requested a full tablet report, sending...
I20260811 00:27:27.849604 13677 ts_manager.cc:194] Registered new tserver with Master: 77d592539883456abd00cb78bf7a2e31 (127.2.120.2:35527)
W20260811 00:27:28.223630 14038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:28.223954 14038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:28.224107 14038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:28.232429 14038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:28.232645 14038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:28.232861 14038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:27:28.243264 14038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:36097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:34851
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:28.244791 14038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:28.246743 14038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:28.261685 14046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:28.261686 14044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:28.262028 14043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:28.262292 14038 server_base.cc:1034] running on GCE node
I20260811 00:27:28.263161 14038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:28.264325 14038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:28.265628 14038 hybrid_clock.cc:648] HybridClock initialized: now 1786408048265549 us; error 80 us; skew 500 ppm
I20260811 00:27:28.266009 14038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:27:28.273514 14038 webserver.cc:460] Webserver started at http://127.2.120.3:37623/ using document root <none> and password file <none>
I20260811 00:27:28.274641 14038 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:28.274822 14038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:28.275310 14038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:27:28.278891 14038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "5ecfcb2058fc4e0b8c9771787013fe00"
format_stamp: "Formatted at 2026-08-11 00:27:28 on dist-test-slave-fd7l"
server_key: "7a76a91619ea69d34f67aa1fef235806"
server_key_iv: "d34c0be39a832f4719c862c850774576"
server_key_version: "encryptionkey@0"
I20260811 00:27:28.279790 14038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "5ecfcb2058fc4e0b8c9771787013fe00"
format_stamp: "Formatted at 2026-08-11 00:27:28 on dist-test-slave-fd7l"
server_key: "7a76a91619ea69d34f67aa1fef235806"
server_key_iv: "d34c0be39a832f4719c862c850774576"
server_key_version: "encryptionkey@0"
I20260811 00:27:28.286135 14038 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 00:27:28.290433 14054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:28.291965 14038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:27:28.292184 14038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "5ecfcb2058fc4e0b8c9771787013fe00"
format_stamp: "Formatted at 2026-08-11 00:27:28 on dist-test-slave-fd7l"
server_key: "7a76a91619ea69d34f67aa1fef235806"
server_key_iv: "d34c0be39a832f4719c862c850774576"
server_key_version: "encryptionkey@0"
I20260811 00:27:28.292407 14038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:28.298909 14038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:28.299979 14038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:28.300320 14038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:28.301507 14038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:27:28.303102 14038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:27:28.303193 14038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:28.303350 14038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:27:28.303442 14038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:28.339529 13887 heartbeater.cc:503] Master 127.2.120.62:34851 was elected leader, sending a full tablet report...
I20260811 00:27:28.342916 14038 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:32853
I20260811 00:27:28.342965 14180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:32853 every 8 connection(s)
I20260811 00:27:28.344445 14038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:27:28.346964 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 14038
I20260811 00:27:28.347185 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:27:28.347800 2528 external_mini_cluster.cc:1428] Setting key 505c833c33c043f9654d8035c509722c
I20260811 00:27:28.355540 14183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:28.348035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:32849 (local address 127.2.120.62:34851)
0811 00:27:28.348636 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:28.348645 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:28.350337 (+ 1692us) server_negotiation.cc:407] Connection header received
0811 00:27:28.352487 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:28.352563 (+ 76us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:28.352716 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:28.352997 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:28.354597 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:28.354602 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:28.354606 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:27:28.354626 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:28.354673 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:27:28.354688 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:28.354834 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:28.355061 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 00:27:28.355156 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":285,"threads_started":1}
I20260811 00:27:28.356767 14184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:28.348859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:34851 (local address 127.2.120.3:32849)
0811 00:27:28.350169 (+ 1310us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:28.350225 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:28.352075 (+ 1850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:28.353322 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:28.353333 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:28.353449 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:28.354220 (+ 771us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:27:28.354223 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:28.354267 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:27:28.354367 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:27:28.354803 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:27:28.354815 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:27:28.354825 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 00:27:28.355252 (+ 427us) client_negotiation.cc:240] Negotiation successful
0811 00:27:28.355578 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":418,"threads_started":1}
I20260811 00:27:28.358505 14181 heartbeater.cc:347] Connected to a master server at 127.2.120.62:34851
I20260811 00:27:28.358909 14181 heartbeater.cc:464] Registering TS with master...
I20260811 00:27:28.359647 14181 heartbeater.cc:511] Master 127.2.120.62:34851 requested a full tablet report, sending...
I20260811 00:27:28.361477 13677 ts_manager.cc:194] Registered new tserver with Master: 5ecfcb2058fc4e0b8c9771787013fe00 (127.2.120.3:32853)
I20260811 00:27:28.363971 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 00:27:28.370296 14183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:27:28.368716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45418 (local address 127.2.120.62:34851)
0811 00:27:28.368892 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:28.368897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:28.369498 (+ 601us) server_negotiation.cc:407] Connection header received
0811 00:27:28.369637 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:28.369641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:28.369697 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:28.369786 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:28.370099 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45418: BlockingRecv error: recv got EOF from 127.0.0.1:45418 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 00:27:28.371899 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 13740
I20260811 00:27:28.376137 14191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:28.372965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51340 (local address 127.2.120.1:40753)
0811 00:27:28.373589 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:28.373597 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:28.373631 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:27:28.373694 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:28.373701 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:28.373929 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:28.374065 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:28.375102 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:28.375109 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:28.375124 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:27:28.375136 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:28.375176 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:27:28.375191 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:28.375345 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:28.375695 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 00:27:28.375823 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":308,"threads_started":1}
I20260811 00:27:28.377815 13881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:27:28.466856 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 13740
I20260811 00:27:28.489593 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 13892
I20260811 00:27:28.494649 14194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:28.491094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.2.120.2:35527)
0811 00:27:28.491740 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:28.491747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:28.491781 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:27:28.491853 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:28.491861 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:28.492109 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:28.492268 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:28.493657 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:28.493666 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:28.493678 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:27:28.493705 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:28.493759 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:27:28.493785 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:28.493924 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:28.494169 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 00:27:28.494311 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":316,"threads_started":1}
I20260811 00:27:28.496162 14029 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:27:28.582790 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 13892
I20260811 00:27:28.603883 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 14038
I20260811 00:27:28.608078 14196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:28.604953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50642 (local address 127.2.120.3:32853)
0811 00:27:28.605625 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:28.605632 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:28.605661 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:27:28.605724 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:28.605731 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:28.605964 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:28.606138 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:28.607308 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:28.607314 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:28.607324 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:27:28.607341 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:28.607378 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:27:28.607392 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:28.607594 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:28.607670 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 00:27:28.607772 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":355,"threads_started":1}
I20260811 00:27:28.609373 14175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:27:28.695351 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 14038
I20260811 00:27:28.716357 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 13641
I20260811 00:27:28.717918 13708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:27:28.829367 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 13641
2026-08-11T00:27:28Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2743 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:27:28 dist-test-slave-fd7l krb5kdc[14202](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:27:28 dist-test-slave-fd7l krb5kdc[14202](info): set up 2 sockets
Aug 11 00:27:28 dist-test-slave-fd7l krb5kdc[14202](info): commencing operation
krb5kdc: starting...
W20260811 00:27:30.113902 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.002s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:27:30 dist-test-slave-fd7l krb5kdc[14202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:27:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:27:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:27:30.429421 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:39903
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:37653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:39903
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:27:30.821102 14215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:30.821465 14215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:30.821640 14215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:30.829232 14215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:27:30.829412 14215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:30.829524 14215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:30.829632 14215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:27:30.829721 14215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:27:30.840147 14215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:39903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:39903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:30.841610 14215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:30.843010 14215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:30.856055 14221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:30.856130 14222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:30.856576 14225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:30.856932 14215 server_base.cc:1034] running on GCE node
I20260811 00:27:30.857981 14215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:30.859690 14215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:30.860975 14215 hybrid_clock.cc:648] HybridClock initialized: now 1786408050860907 us; error 65 us; skew 500 ppm
I20260811 00:27:30.861413 14215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:27:30 dist-test-slave-fd7l krb5kdc[14202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408050, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:27:30.865553 14215 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:27:30.867648 14215 webserver.cc:460] Webserver started at http://127.2.120.62:40091/ using document root <none> and password file <none>
I20260811 00:27:30.868494 14215 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:30.868748 14215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:30.869110 14215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:27:30.872917 14215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "0715701e2a05425a8eefbb25de134fce"
format_stamp: "Formatted at 2026-08-11 00:27:30 on dist-test-slave-fd7l"
server_key: "e1e52f1ee24d126a7c5a8ccd77dcdc7b"
server_key_iv: "63507d688d40019a21111132b763a5c5"
server_key_version: "encryptionkey@0"
I20260811 00:27:30.873919 14215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "0715701e2a05425a8eefbb25de134fce"
format_stamp: "Formatted at 2026-08-11 00:27:30 on dist-test-slave-fd7l"
server_key: "e1e52f1ee24d126a7c5a8ccd77dcdc7b"
server_key_iv: "63507d688d40019a21111132b763a5c5"
server_key_version: "encryptionkey@0"
I20260811 00:27:30.880120 14215 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 00:27:30.884215 14233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:30.886765 14215 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:27:30.887048 14215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "0715701e2a05425a8eefbb25de134fce"
format_stamp: "Formatted at 2026-08-11 00:27:30 on dist-test-slave-fd7l"
server_key: "e1e52f1ee24d126a7c5a8ccd77dcdc7b"
server_key_iv: "63507d688d40019a21111132b763a5c5"
server_key_version: "encryptionkey@0"
I20260811 00:27:30.887331 14215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:30.935737 14215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:30.940711 14215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:30.941035 14215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:30.961772 14215 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:39903
I20260811 00:27:30.961803 14291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:39903 every 8 connection(s)
I20260811 00:27:30.963377 14215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:27:30.967814 14292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:27:30.972663 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 14215
I20260811 00:27:30.972927 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:27:30.973877 2528 external_mini_cluster.cc:1428] Setting key cbcf0534c86738405670a6e75df6f651
I20260811 00:27:30.986793 14292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce: Bootstrap starting.
I20260811 00:27:30.991510 14292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce: Neither blocks nor log segments found. Creating new log.
I20260811 00:27:30.993137 14292 log.cc:824] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:27:30 dist-test-slave-fd7l krb5kdc[14202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:27:30.997669 14292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce: No bootstrap required, opened a new log
I20260811 00:27:31.009497 14295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:30.977062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38110 (local address 127.2.120.62:39903)
0811 00:27:30.978436 (+ 1374us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:30.978453 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:30.978509 (+ 56us) server_negotiation.cc:407] Connection header received
0811 00:27:30.981187 (+ 2678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:30.981231 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:30.981862 (+ 631us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:30.982406 (+ 544us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:30.985479 (+ 3073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:30.987615 (+ 2136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:30.989792 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:30.990971 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:30.991218 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:30.997167 (+ 5949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:30.997179 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:30.997197 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:30.997246 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:31.002682 (+ 5436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:31.004054 (+ 1372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:31.004067 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:31.004080 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:31.004594 (+ 514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:31.005108 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:31.005114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:31.005118 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:31.006341 (+ 1223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:31.006799 (+ 458us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:31.007441 (+ 642us) server_negotiation.cc:299] Negotiation successful
0811 00:27:31.007991 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1098,"thread_start_us":943,"threads_started":1}
I20260811 00:27:31.011569 14292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0715701e2a05425a8eefbb25de134fce" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39903 } }
I20260811 00:27:31.012141 14292 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:27:31.012277 14292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0715701e2a05425a8eefbb25de134fce, State: Initialized, Role: FOLLOWER
I20260811 00:27:31.013128 14292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0715701e2a05425a8eefbb25de134fce" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39903 } }
I20260811 00:27:31.013485 14292 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:27:31.013653 14292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:27:31.013841 14292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:27:31.017781 14292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0715701e2a05425a8eefbb25de134fce" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39903 } }
I20260811 00:27:31.018580 14292 leader_election.cc:302] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0715701e2a05425a8eefbb25de134fce; no voters:
I20260811 00:27:31.019356 14292 leader_election.cc:288] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:27:31.019431 14297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:27:31.020339 14297 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce [term 1 LEADER]: Becoming Leader. State: Replica: 0715701e2a05425a8eefbb25de134fce, State: Running, Role: LEADER
I20260811 00:27:31.021024 14297 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0715701e2a05425a8eefbb25de134fce" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39903 } }
I20260811 00:27:31.022231 14292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:27:31.024360 14299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0715701e2a05425a8eefbb25de134fce. Latest consensus state: current_term: 1 leader_uuid: "0715701e2a05425a8eefbb25de134fce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0715701e2a05425a8eefbb25de134fce" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39903 } } }
I20260811 00:27:31.024376 14298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0715701e2a05425a8eefbb25de134fce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0715701e2a05425a8eefbb25de134fce" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39903 } } }
I20260811 00:27:31.024897 14298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce [sys.catalog]: This master's current role is: LEADER
I20260811 00:27:31.024917 14299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce [sys.catalog]: This master's current role is: LEADER
I20260811 00:27:31.025997 14303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:27:31.032908 14303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:27:31.043053 14316 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 00:27:31.045127 14303 catalog_manager.cc:1347] Generated new cluster ID: c12410dd50f04bd882b9de7d9578efd0
I20260811 00:27:31.045243 14303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:27:31.062404 14303 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:27:31.064316 14303 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:27:31.072572 14303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce: Generated new TSK 0
I20260811 00:27:31.073756 14303 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
I20260811 00:27:31.146335 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:39903
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:37653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:27:31.525692 14321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:31.526029 14321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:31.526186 14321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:31.533834 14321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:31.533973 14321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:31.534153 14321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:27:31.544956 14321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:39903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:31.546434 14321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:31.547895 14321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:31.562850 14327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:31.562795 14330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:31.562898 14328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:31.563916 14321 server_base.cc:1034] running on GCE node
I20260811 00:27:31.564733 14321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:31.565814 14321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:31.567384 14321 hybrid_clock.cc:648] HybridClock initialized: now 1786408051567237 us; error 92 us; skew 500 ppm
I20260811 00:27:31.567919 14321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:27:31 dist-test-slave-fd7l krb5kdc[14202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408051, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:27:31.572568 14321 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:27:31.574347 14321 webserver.cc:460] Webserver started at http://127.2.120.1:34949/ using document root <none> and password file <none>
I20260811 00:27:31.575181 14321 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:31.575350 14321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:31.575742 14321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:27:31.579129 14321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "79ced76333674607841c169b5c9a615e"
format_stamp: "Formatted at 2026-08-11 00:27:31 on dist-test-slave-fd7l"
server_key: "8b6569a13c3436d54b356e5424a8d59f"
server_key_iv: "5d364c1cb72fe37299773354a249cc77"
server_key_version: "encryptionkey@0"
I20260811 00:27:31.580086 14321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "79ced76333674607841c169b5c9a615e"
format_stamp: "Formatted at 2026-08-11 00:27:31 on dist-test-slave-fd7l"
server_key: "8b6569a13c3436d54b356e5424a8d59f"
server_key_iv: "5d364c1cb72fe37299773354a249cc77"
server_key_version: "encryptionkey@0"
I20260811 00:27:31.586313 14321 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.004s
I20260811 00:27:31.590619 14337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:31.592375 14321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:27:31.592633 14321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "79ced76333674607841c169b5c9a615e"
format_stamp: "Formatted at 2026-08-11 00:27:31 on dist-test-slave-fd7l"
server_key: "8b6569a13c3436d54b356e5424a8d59f"
server_key_iv: "5d364c1cb72fe37299773354a249cc77"
server_key_version: "encryptionkey@0"
I20260811 00:27:31.592844 14321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:31.611930 14321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:31.617146 14321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:31.617471 14321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:31.618613 14321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:27:31.620296 14321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:27:31.620398 14321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:31.620580 14321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:27:31.620664 14321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 00:27:31.660562 14321 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:44781
I20260811 00:27:31.660856 14466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:44781 every 8 connection(s)
I20260811 00:27:31.662068 14321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:27:31.668843 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 14321
I20260811 00:27:31.669103 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:27:31.669780 2528 external_mini_cluster.cc:1428] Setting key a14f438b161e1cff611f447e0e82ffb5
Aug 11 00:27:31 dist-test-slave-fd7l krb5kdc[14202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408051, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:27:31.684983 14469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:31.665325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:51271 (local address 127.2.120.62:39903)
0811 00:27:31.665809 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:31.665814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:31.666723 (+ 909us) server_negotiation.cc:407] Connection header received
0811 00:27:31.668788 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:31.668794 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:31.668899 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:31.669026 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:31.670644 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:31.671438 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:31.673230 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:31.673645 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:31.673788 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:31.677640 (+ 3852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:31.677646 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:31.677649 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:31.677686 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:31.680459 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:31.681695 (+ 1236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:31.681703 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:31.681706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:31.681964 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:31.682706 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:31.682712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:31.682715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:31.683057 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:31.683215 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:31.684259 (+ 1044us) server_negotiation.cc:299] Negotiation successful
0811 00:27:31.684679 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":230,"threads_started":1}
I20260811 00:27:31.686400 14470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:31.665754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:39903 (local address 127.2.120.1:51271)
0811 00:27:31.666529 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:31.666575 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:31.668424 (+ 1849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:31.669285 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:31.669293 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:31.669827 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:31.670459 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:31.670477 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:31.671627 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:31.671631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:31.673072 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:31.673086 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:31.673912 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:31.673920 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:31.674167 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:31.674915 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:31.674945 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:31.677366 (+ 2421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:27:31.680676 (+ 3310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:27:31.680686 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:27:31.680706 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:27:31.681328 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:27:31.682159 (+ 831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:27:31.682167 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:27:31.682172 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:27:31.682482 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:27:31.683232 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:27:31.683245 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:27:31.683777 (+ 532us) client_negotiation.cc:769] Sending connection context
0811 00:27:31.684113 (+ 336us) client_negotiation.cc:240] Negotiation successful
0811 00:27:31.684789 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":322,"threads_started":1}
I20260811 00:27:31.688381 14467 heartbeater.cc:347] Connected to a master server at 127.2.120.62:39903
I20260811 00:27:31.689069 14467 heartbeater.cc:464] Registering TS with master...
I20260811 00:27:31.690152 14467 heartbeater.cc:511] Master 127.2.120.62:39903 requested a full tablet report, sending...
I20260811 00:27:31.693219 14251 ts_manager.cc:194] Registered new tserver with Master: 79ced76333674607841c169b5c9a615e (127.2.120.1:44781)
I20260811 00:27:31.694847 14251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:51271
WARNING: no policy specified for kudu/127.2.120.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
I20260811 00:27:31.747285 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:39903
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:37653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:27:32.174520 14474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:32.174922 14474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:32.175087 14474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:32.183708 14474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:32.183905 14474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:32.184103 14474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:27:32.195055 14474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:39903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:32.196722 14474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:32.198374 14474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:32.213589 14479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:32.213652 14480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:32.213693 14484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:32.214385 14474 server_base.cc:1034] running on GCE node
I20260811 00:27:32.215323 14474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:32.216874 14474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:32.218142 14474 hybrid_clock.cc:648] HybridClock initialized: now 1786408052218073 us; error 82 us; skew 500 ppm
I20260811 00:27:32.218629 14474 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:27:32 dist-test-slave-fd7l krb5kdc[14202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408052, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:27:32.222945 14474 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:27:32.224808 14474 webserver.cc:460] Webserver started at http://127.2.120.2:45087/ using document root <none> and password file <none>
I20260811 00:27:32.225607 14474 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:32.225760 14474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:32.226126 14474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:27:32.229837 14474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "c06dfe134d0849009f6125fd9e0c6e39"
format_stamp: "Formatted at 2026-08-11 00:27:32 on dist-test-slave-fd7l"
server_key: "a410892c20a771552b742b1fd26c2902"
server_key_iv: "0e229ab8a6f1ce69c86663f0defb412a"
server_key_version: "encryptionkey@0"
I20260811 00:27:32.230732 14474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "c06dfe134d0849009f6125fd9e0c6e39"
format_stamp: "Formatted at 2026-08-11 00:27:32 on dist-test-slave-fd7l"
server_key: "a410892c20a771552b742b1fd26c2902"
server_key_iv: "0e229ab8a6f1ce69c86663f0defb412a"
server_key_version: "encryptionkey@0"
I20260811 00:27:32.237182 14474 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 00:27:32.241350 14491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:32.243188 14474 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:27:32.243418 14474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "c06dfe134d0849009f6125fd9e0c6e39"
format_stamp: "Formatted at 2026-08-11 00:27:32 on dist-test-slave-fd7l"
server_key: "a410892c20a771552b742b1fd26c2902"
server_key_iv: "0e229ab8a6f1ce69c86663f0defb412a"
server_key_version: "encryptionkey@0"
I20260811 00:27:32.243681 14474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:32.278469 14474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:32.284034 14474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:32.284408 14474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:32.286324 14474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:27:32.288669 14474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:27:32.288789 14474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 00:27:32.288941 14474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:27:32.289008 14474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:32.330435 14474 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:43419
I20260811 00:27:32.330497 14625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:43419 every 8 connection(s)
I20260811 00:27:32.332638 14474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:27:32.334841 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 14474
I20260811 00:27:32.335079 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:27:32.335608 2528 external_mini_cluster.cc:1428] Setting key 8e3aa3060a8d5b7f015e0135f8460328
Aug 11 00:27:32 dist-test-slave-fd7l krb5kdc[14202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408052, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:27:32.363983 14628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:32.337082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:55467 (local address 127.2.120.62:39903)
0811 00:27:32.338431 (+ 1349us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:32.338441 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:32.341820 (+ 3379us) server_negotiation.cc:407] Connection header received
0811 00:27:32.344072 (+ 2252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:32.344078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:32.344212 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:32.344350 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:32.346287 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:32.347096 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:32.349360 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:32.349836 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:32.349995 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:32.354693 (+ 4698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:32.354702 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:32.354706 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:32.354781 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:32.359105 (+ 4324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:32.360433 (+ 1328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:32.360440 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:32.360443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:32.360700 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:32.361267 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:32.361273 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:32.361275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:32.361679 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:32.362148 (+ 469us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:32.363215 (+ 1067us) server_negotiation.cc:299] Negotiation successful
0811 00:27:32.363558 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1204,"thread_start_us":1070,"threads_started":1}
I20260811 00:27:32.365247 14629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:32.338013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:39903 (local address 127.2.120.2:55467)
0811 00:27:32.341439 (+ 3426us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:32.341513 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:32.343697 (+ 2184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:32.344741 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:32.344751 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:32.345392 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:32.346049 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:32.346070 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:32.347308 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:32.347313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:32.349155 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:32.349172 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:32.350165 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:32.350176 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:32.350438 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:32.351282 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:32.351310 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:32.353876 (+ 2566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:27:32.359428 (+ 5552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:27:32.359442 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:27:32.359468 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:27:32.360224 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:27:32.360841 (+ 617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:27:32.360847 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:27:32.360851 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:27:32.361111 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:27:32.362098 (+ 987us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:27:32.362125 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:27:32.362689 (+ 564us) client_negotiation.cc:769] Sending connection context
0811 00:27:32.363055 (+ 366us) client_negotiation.cc:240] Negotiation successful
0811 00:27:32.363743 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2871,"thread_start_us":2563,"threads_started":1}
WARNING: no policy specified for kudu/127.2.120.3@KRBTEST.COM; defaulting to no policy
I20260811 00:27:32.367123 14626 heartbeater.cc:347] Connected to a master server at 127.2.120.62:39903
I20260811 00:27:32.367511 14626 heartbeater.cc:464] Registering TS with master...
I20260811 00:27:32.368391 14626 heartbeater.cc:511] Master 127.2.120.62:39903 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.3@KRBTEST.COM" created.
I20260811 00:27:32.370424 14251 ts_manager.cc:194] Registered new tserver with Master: c06dfe134d0849009f6125fd9e0c6e39 (127.2.120.2:43419)
I20260811 00:27:32.371906 14251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:55467
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
I20260811 00:27:32.408469 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:39903
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:37653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:27:32.698577 14467 heartbeater.cc:503] Master 127.2.120.62:39903 was elected leader, sending a full tablet report...
W20260811 00:27:32.787894 14632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:27:32.788205 14632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:27:32.788302 14632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:27:32.796159 14632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:27:32.796329 14632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:27:32.796561 14632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:27:32.806924 14632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:39903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:27:32.808372 14632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:27:32.809779 14632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:27:32.823938 14638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:32.823997 14641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:27:32.823988 14639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:27:32.824617 14632 server_base.cc:1034] running on GCE node
I20260811 00:27:32.825342 14632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:27:32.826385 14632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:27:32.827607 14632 hybrid_clock.cc:648] HybridClock initialized: now 1786408052827527 us; error 60 us; skew 500 ppm
I20260811 00:27:32.827946 14632 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:27:32 dist-test-slave-fd7l krb5kdc[14202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408052, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:27:32.831481 14632 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:27:32.833411 14632 webserver.cc:460] Webserver started at http://127.2.120.3:43877/ using document root <none> and password file <none>
I20260811 00:27:32.834204 14632 fs_manager.cc:359] Metadata directory not provided
I20260811 00:27:32.834367 14632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:27:32.834717 14632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:27:32.838047 14632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "e15ba402a91e4243b7e12f3e6521092a"
format_stamp: "Formatted at 2026-08-11 00:27:32 on dist-test-slave-fd7l"
server_key: "b4d09d7d2b6ebcab70004f3d9bea4db7"
server_key_iv: "9a6a04c07800c93af85e3f5ed4a9f556"
server_key_version: "encryptionkey@0"
I20260811 00:27:32.838934 14632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "e15ba402a91e4243b7e12f3e6521092a"
format_stamp: "Formatted at 2026-08-11 00:27:32 on dist-test-slave-fd7l"
server_key: "b4d09d7d2b6ebcab70004f3d9bea4db7"
server_key_iv: "9a6a04c07800c93af85e3f5ed4a9f556"
server_key_version: "encryptionkey@0"
I20260811 00:27:32.844851 14632 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 00:27:32.848930 14649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:32.850556 14632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:27:32.850780 14632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "e15ba402a91e4243b7e12f3e6521092a"
format_stamp: "Formatted at 2026-08-11 00:27:32 on dist-test-slave-fd7l"
server_key: "b4d09d7d2b6ebcab70004f3d9bea4db7"
server_key_iv: "9a6a04c07800c93af85e3f5ed4a9f556"
server_key_version: "encryptionkey@0"
I20260811 00:27:32.851052 14632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:27:32.878877 14632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:27:32.884452 14632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:27:32.884886 14632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:27:32.886014 14632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:27:32.887636 14632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:27:32.887753 14632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:32.887920 14632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:27:32.887996 14632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:27:32.928454 14632 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:44511
I20260811 00:27:32.928601 14772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:44511 every 8 connection(s)
I20260811 00:27:32.929970 14632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:27:32.932601 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 14632
I20260811 00:27:32.932825 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:27:32.933362 2528 external_mini_cluster.cc:1428] Setting key 9efab757014496815a2a6517b1c0679d
Aug 11 00:27:32 dist-test-slave-fd7l krb5kdc[14202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408052, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:27:32.958458 14776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:32.934107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:45437 (local address 127.2.120.62:39903)
0811 00:27:32.934801 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:32.934807 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:32.936318 (+ 1511us) server_negotiation.cc:407] Connection header received
0811 00:27:32.939112 (+ 2794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:32.939120 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:32.939261 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:32.939402 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:32.941906 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:32.942735 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:32.944460 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:32.944953 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:32.945105 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:32.949472 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:32.949480 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:32.949486 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:32.949545 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:32.953547 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:32.954764 (+ 1217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:32.954774 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:32.954778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:32.955116 (+ 338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:32.956187 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:32.956197 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:32.956200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:32.956703 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:32.956869 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:32.957860 (+ 991us) server_negotiation.cc:299] Negotiation successful
0811 00:27:32.958101 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":303,"threads_started":1}
I20260811 00:27:32.959481 14777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:32.935066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:39903 (local address 127.2.120.3:45437)
0811 00:27:32.936115 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:32.936185 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:32.938752 (+ 2567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:32.939724 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:32.939733 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:32.940919 (+ 1186us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:27:32.941618 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:32.941640 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:32.942938 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:32.942943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:32.944284 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:32.944299 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:32.945220 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:32.945228 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:32.945505 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:32.946331 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:27:32.946357 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:27:32.949166 (+ 2809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:27:32.953805 (+ 4639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:27:32.953815 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:27:32.953835 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:27:32.954419 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:27:32.955559 (+ 1140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:27:32.955565 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:27:32.955568 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:27:32.955940 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:27:32.956922 (+ 982us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:27:32.956930 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:27:32.957353 (+ 423us) client_negotiation.cc:769] Sending connection context
0811 00:27:32.957663 (+ 310us) client_negotiation.cc:240] Negotiation successful
0811 00:27:32.958142 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":421,"threads_started":1}
I20260811 00:27:32.961742 14773 heartbeater.cc:347] Connected to a master server at 127.2.120.62:39903
I20260811 00:27:32.962195 14773 heartbeater.cc:464] Registering TS with master...
I20260811 00:27:32.963402 14773 heartbeater.cc:511] Master 127.2.120.62:39903 requested a full tablet report, sending...
I20260811 00:27:32.965332 14251 ts_manager.cc:194] Registered new tserver with Master: e15ba402a91e4243b7e12f3e6521092a (127.2.120.3:44511)
I20260811 00:27:32.966540 14251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:45437
I20260811 00:27:32.979143 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:27:33.012864 14776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:32.984997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38112 (local address 127.2.120.62:39903)
0811 00:27:32.985458 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:32.985469 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:32.985926 (+ 457us) server_negotiation.cc:407] Connection header received
0811 00:27:32.986244 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:32.986249 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:32.986382 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:32.986516 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:32.988592 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:32.989737 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:32.992373 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:32.993367 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:32.993725 (+ 358us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:32.999322 (+ 5597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:32.999336 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:32.999340 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:32.999457 (+ 117us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:33.005723 (+ 6266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:33.007815 (+ 2092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:33.007829 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:33.007832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:33.008241 (+ 409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:33.009304 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:33.009313 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:33.009316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:33.010162 (+ 846us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:33.010718 (+ 556us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:33.011654 (+ 936us) server_negotiation.cc:299] Negotiation successful
0811 00:27:33.012181 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260811 00:27:33.025821 14251 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:27:33.037925 14251 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:27:33.072281 14787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:33.057623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:44511 (local address 127.0.0.1:46758)
0811 00:27:33.059051 (+ 1428us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:33.059117 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:33.059424 (+ 307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:33.060860 (+ 1436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:33.060873 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:33.060920 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:27:33.061505 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:33.061523 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.064456 (+ 2933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.064465 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:33.068054 (+ 3589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:33.068095 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.071247 (+ 3152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.071252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:33.071430 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:33.071467 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 00:27:33.071638 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 00:27:33.071863 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":369,"threads_started":1}
I20260811 00:27:33.072281 14788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:33.058069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46758 (local address 127.2.120.3:44511)
0811 00:27:33.058913 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:33.058923 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:33.059234 (+ 311us) server_negotiation.cc:407] Connection header received
0811 00:27:33.059748 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:33.059761 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:33.060125 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:33.060334 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:27:33.061801 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.064146 (+ 2345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.068410 (+ 4264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.070553 (+ 2143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.070704 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:33.071041 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:33.071709 (+ 668us) server_negotiation.cc:299] Negotiation successful
0811 00:27:33.071900 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":449,"threads_started":1}
I20260811 00:27:33.077101 14702 tablet_service.cc:1467] Processing CreateTablet for tablet df7c3772c05e417e869b828b3b5489df (DEFAULT_TABLE table=test-table [id=339f9a88d93a4029b5737f174b82a72e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:27:33.078997 14702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df7c3772c05e417e869b828b3b5489df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:27:33.082860 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:33.059170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41390 (local address 127.2.120.1:44781)
0811 00:27:33.071388 (+ 12218us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:33.071402 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:33.071447 (+ 45us) server_negotiation.cc:407] Connection header received
0811 00:27:33.071602 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:33.071616 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:33.072011 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:33.073033 (+ 1022us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:27:33.073340 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.077568 (+ 4228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.080186 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.081561 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.081741 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:33.081997 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:33.082310 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 00:27:33.082500 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11947,"thread_start_us":11723,"threads_started":1}
I20260811 00:27:33.082912 14790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:33.058552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:44781 (local address 127.0.0.1:41390)
0811 00:27:33.059372 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:33.059394 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:33.059537 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:33.072363 (+ 12826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:33.072371 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:33.072414 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:27:33.073012 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:33.073029 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.077724 (+ 4695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.077730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:33.079991 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:33.080005 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.081836 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.081843 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:33.082082 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:33.082110 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 00:27:33.082255 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 00:27:33.082452 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"mutex_wait_us":33,"thread_start_us":839,"threads_started":1}
I20260811 00:27:33.083613 14791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:33.058909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:43419 (local address 127.0.0.1:45096)
0811 00:27:33.068529 (+ 9620us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:33.068561 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:33.068760 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:33.075399 (+ 6639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:33.075406 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:33.075434 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:27:33.075808 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:33.075819 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.077343 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.077349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:33.079911 (+ 2562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:33.079933 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.082292 (+ 2359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.082298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:33.082530 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:33.082569 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 00:27:33.082675 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 00:27:33.082830 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9340,"mutex_wait_us":1,"thread_start_us":9974,"threads_started":1}
I20260811 00:27:33.084986 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:33.059513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45096 (local address 127.2.120.2:43419)
0811 00:27:33.074849 (+ 15336us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:33.074858 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:33.074889 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:27:33.075001 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:33.075007 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:33.075196 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:33.075416 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:27:33.075974 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.077190 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.080132 (+ 2942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.081745 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.081884 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:33.082085 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:33.083360 (+ 1275us) server_negotiation.cc:299] Negotiation successful
0811 00:27:33.084143 (+ 783us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15005,"thread_start_us":14866,"threads_started":1}
I20260811 00:27:33.087488 14392 tablet_service.cc:1467] Processing CreateTablet for tablet df7c3772c05e417e869b828b3b5489df (DEFAULT_TABLE table=test-table [id=339f9a88d93a4029b5737f174b82a72e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:27:33.090013 14392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df7c3772c05e417e869b828b3b5489df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:27:33.092119 14548 tablet_service.cc:1467] Processing CreateTablet for tablet df7c3772c05e417e869b828b3b5489df (DEFAULT_TABLE table=test-table [id=339f9a88d93a4029b5737f174b82a72e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:27:33.093945 14548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df7c3772c05e417e869b828b3b5489df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:27:33.094901 14794 tablet_bootstrap.cc:492] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a: Bootstrap starting.
I20260811 00:27:33.099800 14794 tablet_bootstrap.cc:654] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a: Neither blocks nor log segments found. Creating new log.
I20260811 00:27:33.101626 14794 log.cc:824] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a: Log is configured to *not* fsync() on all Append() calls
I20260811 00:27:33.106009 14794 tablet_bootstrap.cc:492] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a: No bootstrap required, opened a new log
I20260811 00:27:33.106395 14794 ts_tablet_manager.cc:1397] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.008s
I20260811 00:27:33.111150 14796 tablet_bootstrap.cc:492] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e: Bootstrap starting.
I20260811 00:27:33.113451 14798 tablet_bootstrap.cc:492] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39: Bootstrap starting.
I20260811 00:27:33.116207 14796 tablet_bootstrap.cc:654] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e: Neither blocks nor log segments found. Creating new log.
I20260811 00:27:33.117878 14798 tablet_bootstrap.cc:654] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39: Neither blocks nor log segments found. Creating new log.
I20260811 00:27:33.118336 14796 log.cc:824] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e: Log is configured to *not* fsync() on all Append() calls
I20260811 00:27:33.119513 14798 log.cc:824] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39: Log is configured to *not* fsync() on all Append() calls
I20260811 00:27:33.120005 14794 raft_consensus.cc:348] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e15ba402a91e4243b7e12f3e6521092a" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 44511 } } peers { permanent_uuid: "c06dfe134d0849009f6125fd9e0c6e39" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43419 } } peers { permanent_uuid: "79ced76333674607841c169b5c9a615e" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44781 } }
I20260811 00:27:33.120685 14794 raft_consensus.cc:374] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:27:33.120821 14794 raft_consensus.cc:729] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e15ba402a91e4243b7e12f3e6521092a, State: Initialized, Role: FOLLOWER
I20260811 00:27:33.121775 14794 consensus_queue.cc:260] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e15ba402a91e4243b7e12f3e6521092a" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 44511 } } peers { permanent_uuid: "c06dfe134d0849009f6125fd9e0c6e39" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43419 } } peers { permanent_uuid: "79ced76333674607841c169b5c9a615e" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44781 } }
I20260811 00:27:33.124441 14798 tablet_bootstrap.cc:492] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39: No bootstrap required, opened a new log
I20260811 00:27:33.124439 14796 tablet_bootstrap.cc:492] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e: No bootstrap required, opened a new log
I20260811 00:27:33.124821 14796 ts_tablet_manager.cc:1397] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.003s
I20260811 00:27:33.124821 14798 ts_tablet_manager.cc:1397] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.010s
I20260811 00:27:33.129257 14773 heartbeater.cc:503] Master 127.2.120.62:39903 was elected leader, sending a full tablet report...
I20260811 00:27:33.130136 14794 ts_tablet_manager.cc:1428] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a: Time spent starting tablet: real 0.024s user 0.009s sys 0.014s
I20260811 00:27:33.138235 14798 raft_consensus.cc:348] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e15ba402a91e4243b7e12f3e6521092a" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 44511 } } peers { permanent_uuid: "c06dfe134d0849009f6125fd9e0c6e39" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43419 } } peers { permanent_uuid: "79ced76333674607841c169b5c9a615e" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44781 } }
I20260811 00:27:33.138235 14796 raft_consensus.cc:348] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e15ba402a91e4243b7e12f3e6521092a" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 44511 } } peers { permanent_uuid: "c06dfe134d0849009f6125fd9e0c6e39" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43419 } } peers { permanent_uuid: "79ced76333674607841c169b5c9a615e" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44781 } }
I20260811 00:27:33.138986 14796 raft_consensus.cc:374] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:27:33.138986 14798 raft_consensus.cc:374] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:27:33.139122 14796 raft_consensus.cc:729] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ced76333674607841c169b5c9a615e, State: Initialized, Role: FOLLOWER
I20260811 00:27:33.139127 14798 raft_consensus.cc:729] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c06dfe134d0849009f6125fd9e0c6e39, State: Initialized, Role: FOLLOWER
I20260811 00:27:33.140108 14796 consensus_queue.cc:260] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e15ba402a91e4243b7e12f3e6521092a" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 44511 } } peers { permanent_uuid: "c06dfe134d0849009f6125fd9e0c6e39" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43419 } } peers { permanent_uuid: "79ced76333674607841c169b5c9a615e" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44781 } }
I20260811 00:27:33.140105 14798 consensus_queue.cc:260] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e15ba402a91e4243b7e12f3e6521092a" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 44511 } } peers { permanent_uuid: "c06dfe134d0849009f6125fd9e0c6e39" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43419 } } peers { permanent_uuid: "79ced76333674607841c169b5c9a615e" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44781 } }
I20260811 00:27:33.142391 14626 heartbeater.cc:503] Master 127.2.120.62:39903 was elected leader, sending a full tablet report...
I20260811 00:27:33.143154 14796 ts_tablet_manager.cc:1428] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e: Time spent starting tablet: real 0.018s user 0.015s sys 0.001s
I20260811 00:27:33.143325 14798 ts_tablet_manager.cc:1428] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39: Time spent starting tablet: real 0.018s user 0.016s sys 0.002s
W20260811 00:27:33.165316 14468 tablet.cc:2369] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:27:33.182246 14775 tablet.cc:2369] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:27:33.305913 14803 raft_consensus.cc:482] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:27:33.306293 14803 raft_consensus.cc:504] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e15ba402a91e4243b7e12f3e6521092a" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 44511 } } peers { permanent_uuid: "c06dfe134d0849009f6125fd9e0c6e39" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43419 } } peers { permanent_uuid: "79ced76333674607841c169b5c9a615e" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44781 } }
I20260811 00:27:33.308429 14803 leader_election.cc:288] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c06dfe134d0849009f6125fd9e0c6e39 (127.2.120.2:43419), 79ced76333674607841c169b5c9a615e (127.2.120.1:44781)
I20260811 00:27:33.318060 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:33.309146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:53769 (local address 127.2.120.2:43419)
0811 00:27:33.309583 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:33.309591 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:33.310194 (+ 603us) server_negotiation.cc:407] Connection header received
0811 00:27:33.310485 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:33.310491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:33.310572 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:33.310708 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:27:33.311681 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.312824 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.315574 (+ 2750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.316945 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.317129 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:33.317232 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:33.317722 (+ 490us) server_negotiation.cc:299] Negotiation successful
0811 00:27:33.317843 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192}
I20260811 00:27:33.318156 14777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:33.309193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:43419 (local address 127.2.120.3:53769)
0811 00:27:33.309989 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:33.310019 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:33.310288 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:33.311100 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:33.311108 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:33.311158 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:27:33.311429 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:33.311443 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.313068 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.313076 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:33.315300 (+ 2224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:33.315322 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.317209 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.317217 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:33.317463 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:33.317503 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 00:27:33.317638 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 00:27:33.317815 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":39}
I20260811 00:27:33.319537 14806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:33.310065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:44781 (local address 127.2.120.3:50893)
0811 00:27:33.311468 (+ 1403us) negotiation.cc:107] Waiting for socket to connect
0811 00:27:33.311490 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:27:33.311626 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:27:33.311993 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:27:33.311998 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:27:33.312029 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:27:33.312413 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:33.312425 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.314631 (+ 2206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.314637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:33.316901 (+ 2264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:27:33.316929 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.318855 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.318869 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:27:33.319011 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:33.319038 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 00:27:33.319125 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 00:27:33.319227 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1267,"mutex_wait_us":1,"thread_start_us":935,"threads_started":1}
I20260811 00:27:33.319629 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:33.309660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:50893 (local address 127.2.120.1:44781)
0811 00:27:33.310019 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:33.310027 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:33.311608 (+ 1581us) server_negotiation.cc:407] Connection header received
0811 00:27:33.311753 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:33.311762 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:33.311848 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:33.311930 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:27:33.312802 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.314340 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.317213 (+ 2873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.318614 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.318750 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:33.318870 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:33.319196 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 00:27:33.319354 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260811 00:27:33.319526 14572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df7c3772c05e417e869b828b3b5489df" candidate_uuid: "e15ba402a91e4243b7e12f3e6521092a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c06dfe134d0849009f6125fd9e0c6e39" is_pre_election: true
I20260811 00:27:33.320240 14572 raft_consensus.cc:2454] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e15ba402a91e4243b7e12f3e6521092a in term 0.
I20260811 00:27:33.320940 14414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df7c3772c05e417e869b828b3b5489df" candidate_uuid: "e15ba402a91e4243b7e12f3e6521092a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79ced76333674607841c169b5c9a615e" is_pre_election: true
I20260811 00:27:33.321216 14653 leader_election.cc:302] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06dfe134d0849009f6125fd9e0c6e39, e15ba402a91e4243b7e12f3e6521092a; no voters:
I20260811 00:27:33.321560 14414 raft_consensus.cc:2454] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e15ba402a91e4243b7e12f3e6521092a in term 0.
I20260811 00:27:33.321731 14803 raft_consensus.cc:2781] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:27:33.321892 14803 raft_consensus.cc:482] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:27:33.322046 14803 raft_consensus.cc:3037] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:27:33.324962 14803 raft_consensus.cc:504] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e15ba402a91e4243b7e12f3e6521092a" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 44511 } } peers { permanent_uuid: "c06dfe134d0849009f6125fd9e0c6e39" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43419 } } peers { permanent_uuid: "79ced76333674607841c169b5c9a615e" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44781 } }
I20260811 00:27:33.325785 14803 leader_election.cc:288] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [CANDIDATE]: Term 1 election: Requested vote from peers c06dfe134d0849009f6125fd9e0c6e39 (127.2.120.2:43419), 79ced76333674607841c169b5c9a615e (127.2.120.1:44781)
I20260811 00:27:33.326442 14414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df7c3772c05e417e869b828b3b5489df" candidate_uuid: "e15ba402a91e4243b7e12f3e6521092a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79ced76333674607841c169b5c9a615e"
I20260811 00:27:33.326467 14572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df7c3772c05e417e869b828b3b5489df" candidate_uuid: "e15ba402a91e4243b7e12f3e6521092a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c06dfe134d0849009f6125fd9e0c6e39"
I20260811 00:27:33.326699 14414 raft_consensus.cc:3037] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:27:33.326741 14572 raft_consensus.cc:3037] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:27:33.329519 14414 raft_consensus.cc:2454] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e15ba402a91e4243b7e12f3e6521092a in term 1.
I20260811 00:27:33.329543 14572 raft_consensus.cc:2454] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e15ba402a91e4243b7e12f3e6521092a in term 1.
I20260811 00:27:33.330174 14653 leader_election.cc:302] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c06dfe134d0849009f6125fd9e0c6e39, e15ba402a91e4243b7e12f3e6521092a; no voters:
I20260811 00:27:33.330588 14803 raft_consensus.cc:2781] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:27:33.331178 14803 raft_consensus.cc:686] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [term 1 LEADER]: Becoming Leader. State: Replica: e15ba402a91e4243b7e12f3e6521092a, State: Running, Role: LEADER
I20260811 00:27:33.331789 14803 consensus_queue.cc:237] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e15ba402a91e4243b7e12f3e6521092a" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 44511 } } peers { permanent_uuid: "c06dfe134d0849009f6125fd9e0c6e39" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43419 } } peers { permanent_uuid: "79ced76333674607841c169b5c9a615e" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44781 } }
I20260811 00:27:33.337749 14250 catalog_manager.cc:5495] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a reported cstate change: term changed from 0 to 1, leader changed from <none> to e15ba402a91e4243b7e12f3e6521092a (127.2.120.3). New cstate: current_term: 1 leader_uuid: "e15ba402a91e4243b7e12f3e6521092a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e15ba402a91e4243b7e12f3e6521092a" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 44511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06dfe134d0849009f6125fd9e0c6e39" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79ced76333674607841c169b5c9a615e" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44781 } health_report { overall_health: UNKNOWN } } }
W20260811 00:27:33.339104 14627 tablet.cc:2369] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:27:33.385564 14788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:33.378931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44126 (local address 127.2.120.3:44511)
0811 00:27:33.379118 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:33.379123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:33.379552 (+ 429us) server_negotiation.cc:407] Connection header received
0811 00:27:33.379781 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:33.379785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:33.379879 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:33.380043 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:27:33.380542 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.381292 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.382867 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.383295 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.383406 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:33.384016 (+ 610us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:27:33.384742 (+ 726us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:27:33.384870 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:33.385247 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 00:27:33.385333 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 00:27:33.394654 14572 raft_consensus.cc:1261] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39 [term 1 FOLLOWER]: Refusing update from remote peer e15ba402a91e4243b7e12f3e6521092a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:27:33.394654 14414 raft_consensus.cc:1261] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e [term 1 FOLLOWER]: Refusing update from remote peer e15ba402a91e4243b7e12f3e6521092a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:27:33.396023 14803 consensus_queue.cc:1035] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06dfe134d0849009f6125fd9e0c6e39" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:27:33.396409 14807 consensus_queue.cc:1035] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79ced76333674607841c169b5c9a615e" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 44781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:27:33.412413 14811 mvcc.cc:205] Tried to move back new op lower bound from 7317127386692046848 to 7317127386451980288. Current Snapshot: MvccSnapshot[applied={T|T < 7317127386692046848}]
I20260811 00:27:33.422559 14812 mvcc.cc:205] Tried to move back new op lower bound from 7317127386692046848 to 7317127386451980288. Current Snapshot: MvccSnapshot[applied={T|T < 7317127386692046848}]
I20260811 00:27:33.422806 14250 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:27:33.423133 14250 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:27:33.431656 14250 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0715701e2a05425a8eefbb25de134fce: Sending DeleteTablet for 3 replicas of tablet df7c3772c05e417e869b828b3b5489df
I20260811 00:27:33.433324 14392 tablet_service.cc:1514] Processing DeleteTablet for tablet df7c3772c05e417e869b828b3b5489df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:27:33 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:41390
I20260811 00:27:33.434772 14548 tablet_service.cc:1514] Processing DeleteTablet for tablet df7c3772c05e417e869b828b3b5489df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:27:33 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:45096
I20260811 00:27:33.434973 14702 tablet_service.cc:1514] Processing DeleteTablet for tablet df7c3772c05e417e869b828b3b5489df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:27:33 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:46758
I20260811 00:27:33.438473 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 14321
I20260811 00:27:33.451418 14821 tablet_replica.cc:330] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39: stopping tablet replica
I20260811 00:27:33.453054 14822 tablet_replica.cc:330] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a: stopping tablet replica
I20260811 00:27:33.454027 14822 raft_consensus.cc:2229] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:27:33.475816 14820 tablet_replica.cc:330] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e: stopping tablet replica
I20260811 00:27:33.476395 14820 raft_consensus.cc:2229] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:27:33.480286 14820 raft_consensus.cc:2258] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:27:33.481122 14821 raft_consensus.cc:2229] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:27:33.482291 14821 raft_consensus.cc:2258] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:27:33.488032 14821 ts_tablet_manager.cc:1905] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:27:33.494236 14820 ts_tablet_manager.cc:1905] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:27:33.500047 14821 ts_tablet_manager.cc:1918] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:27:33.500311 14821 log.cc:1196] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/df7c3772c05e417e869b828b3b5489df
Aug 11 00:27:33 dist-test-slave-fd7l krb5kdc[14202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:27:33.501026 14821 ts_tablet_manager.cc:1939] T df7c3772c05e417e869b828b3b5489df P c06dfe134d0849009f6125fd9e0c6e39: Deleting consensus metadata
I20260811 00:27:33.504223 14822 raft_consensus.cc:2258] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:27:33.504534 14238 catalog_manager.cc:4841] TS c06dfe134d0849009f6125fd9e0c6e39 (127.2.120.2:43419): tablet df7c3772c05e417e869b828b3b5489df (table test-table [id=339f9a88d93a4029b5737f174b82a72e]) successfully deleted
I20260811 00:27:33.507187 14822 ts_tablet_manager.cc:1905] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:27:33.510046 14820 ts_tablet_manager.cc:1918] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:27:33.510272 14820 log.cc:1196] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/df7c3772c05e417e869b828b3b5489df
I20260811 00:27:33.510842 14820 ts_tablet_manager.cc:1939] T df7c3772c05e417e869b828b3b5489df P 79ced76333674607841c169b5c9a615e: Deleting consensus metadata
I20260811 00:27:33.511243 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:33.440026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33628 (local address 127.2.120.1:44781)
0811 00:27:33.440229 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:33.440235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:33.477156 (+ 36921us) server_negotiation.cc:407] Connection header received
0811 00:27:33.477594 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:33.477600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:33.477692 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:33.477784 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:33.486295 (+ 8511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.487482 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.489801 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.490491 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.490623 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:33.502031 (+ 11408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:33.502055 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:33.502079 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:33.502167 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:33.506938 (+ 4771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:33.508054 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:33.508064 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:33.508068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:33.508286 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:33.508859 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:33.508865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:33.508869 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:33.509394 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:33.509735 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:33.510404 (+ 669us) server_negotiation.cc:299] Negotiation successful
0811 00:27:33.510738 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:27:33.512920 14462 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:27:33.513463 14236 catalog_manager.cc:4841] TS 79ced76333674607841c169b5c9a615e (127.2.120.1:44781): tablet df7c3772c05e417e869b828b3b5489df (table test-table [id=339f9a88d93a4029b5737f174b82a72e]) successfully deleted
I20260811 00:27:33.517181 14822 ts_tablet_manager.cc:1918] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:27:33.517446 14822 log.cc:1196] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/df7c3772c05e417e869b828b3b5489df
I20260811 00:27:33.518098 14822 ts_tablet_manager.cc:1939] T df7c3772c05e417e869b828b3b5489df P e15ba402a91e4243b7e12f3e6521092a: Deleting consensus metadata
I20260811 00:27:33.520273 14238 catalog_manager.cc:4841] TS e15ba402a91e4243b7e12f3e6521092a (127.2.120.3:44511): tablet df7c3772c05e417e869b828b3b5489df (table test-table [id=339f9a88d93a4029b5737f174b82a72e]) successfully deleted
I20260811 00:27:33.609906 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 14321
I20260811 00:27:33.634356 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 14474
Aug 11 00:27:33 dist-test-slave-fd7l krb5kdc[14202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.2@KRBTEST.COM
I20260811 00:27:33.661806 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:33.635417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47454 (local address 127.2.120.2:43419)
0811 00:27:33.635633 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:33.635638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:33.635855 (+ 217us) server_negotiation.cc:407] Connection header received
0811 00:27:33.636272 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:33.636278 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:33.636401 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:33.636626 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:33.638685 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.639693 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.642151 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.642626 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.642762 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:33.650362 (+ 7600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:33.650389 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:33.650415 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:33.650484 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:33.656155 (+ 5671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:33.658087 (+ 1932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:33.658105 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:33.658109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:33.658460 (+ 351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:33.659043 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:33.659049 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:33.659052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:33.659757 (+ 705us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:33.660280 (+ 523us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:33.660792 (+ 512us) server_negotiation.cc:299] Negotiation successful
0811 00:27:33.661236 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 00:27:33.662998 14620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:27:33.781832 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 14474
I20260811 00:27:33.806428 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 14632
Aug 11 00:27:33 dist-test-slave-fd7l krb5kdc[14202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.3@KRBTEST.COM
I20260811 00:27:33.826900 14788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:27:33.807430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44138 (local address 127.2.120.3:44511)
0811 00:27:33.807616 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 00:27:33.807621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:27:33.807771 (+ 150us) server_negotiation.cc:407] Connection header received
0811 00:27:33.807999 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:27:33.808003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:27:33.808103 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:27:33.808207 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:27:33.810172 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.811112 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.813087 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:27:33.813567 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:27:33.813682 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:27:33.819850 (+ 6168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:27:33.819866 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:27:33.819887 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:27:33.819951 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:27:33.823490 (+ 3539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:33.824685 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:33.824693 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:33.824696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:33.824861 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:27:33.825337 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:27:33.825340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:27:33.825342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:27:33.825721 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:27:33.825973 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 00:27:33.826386 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 00:27:33.826573 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 00:27:33.827728 14768 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:27:33.957127 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 14632
I20260811 00:27:33.982527 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 14215
I20260811 00:27:33.983819 14286 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:27:34.098227 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 14215
2026-08-11T00:27:34Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (5273 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:27:34 dist-test-slave-fd7l krb5kdc[14831](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:27:34 dist-test-slave-fd7l krb5kdc[14831](info): set up 2 sockets
Aug 11 00:27:34 dist-test-slave-fd7l krb5kdc[14831](info): commencing operation
krb5kdc: starting...
W20260811 00:27:35.438701 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.005s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:27:35 dist-test-slave-fd7l krb5kdc[14831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:27:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:27:35Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.120.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.120.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.120.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/rangeradmin_127.2.120.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.120.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/rangeradmin_127.2.120.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.120.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.120.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.120.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/rangerlookup_127.2.120.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.120.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/rangerlookup_127.2.120.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.120.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.120.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.120.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/HTTP_127.2.120.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.120.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/HTTP_127.2.120.20@KRBTEST.COM.keytab.
I20260811 00:27:35.897738 2528 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskY6MNab/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/postgres -l logfile start
2026-08-11 00:27:38.983 UTC [14868] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 00:27:38.983 UTC [14868] LOG: listening on IPv4 address "127.2.120.20", port 56291
2026-08-11 00:27:38.988 UTC [14868] LOG: listening on Unix socket "/tmp/.s.PGSQL.56291"
2026-08-11 00:27:39.016 UTC [14872] LOG: database system was shut down at 2026-08-11 00:27:37 UTC
2026-08-11 00:27:39.020 UTC [14868] LOG: database system is ready to accept connections
I20260811 00:27:40.176366 2528 mini_postgres.cc:96] Postgres bound to 56291
2026-08-11 00:27:40.202 UTC [14883] FATAL: database "slave" does not exist
127.2.120.20:56291 - accepting connections
I20260811 00:27:40.204052 2528 mini_ranger.cc:161] Starting Ranger...
I20260811 00:27:40.240484 2528 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 00:27:40.879150 2528 mini_ranger.cc:87] Created ranger Postgres database
I20260811 00:27:40.879307 2528 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ranger-admin
2026-08-11 00:27:41,373 [I] DB FLAVOR :POSTGRES
2026-08-11 00:27:41,373 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 00:27:41,373 [I] Checking connection..
2026-08-11 00:27:41,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 00:27:41,669 [I] Checking connection passed.
2026-08-11 00:27:41,670 [I] --------- Verifying version history table ---------
2026-08-11 00:27:41,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 00:27:41,955 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 00:27:41,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 00:27:42,261 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 00:27:42,261 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 00:27:42,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 00:27:42.539 UTC [14954] WARNING: there is no transaction in progress
2026-08-11 00:27:42,548 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 00:27:42,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 00:27:42,830 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 00:27:42,831 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 00:27:42,831 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 00:27:43,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 00:27:43,391 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-fd7l.c.gcp-upstream.internal','N') ;"
2026-08-11 00:27:43,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 00:27:43,939 [I] Table x_portal_user does not exist in database ranger
2026-08-11 00:27:43,939 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 00:27:44,232 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 00:27:44,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 00:27:44,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 00:27:44,793 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 00:27:44,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 00:27:45.976 UTC [15080] WARNING: there is no transaction in progress
2026-08-11 00:27:45.994 UTC [15080] WARNING: there is no transaction in progress
2026-08-11 00:27:46.015 UTC [15080] WARNING: there is no transaction in progress
2026-08-11 00:27:46.037 UTC [15080] WARNING: there is no transaction in progress
2026-08-11 00:27:46.059 UTC [15080] WARNING: there is no transaction in progress
2026-08-11 00:27:46.081 UTC [15080] WARNING: there is no transaction in progress
2026-08-11 00:27:46.149 UTC [15080] WARNING: there is no transaction in progress
2026-08-11 00:27:46.158 UTC [15080] WARNING: there is no transaction in progress
2026-08-11 00:27:46.178 UTC [15080] WARNING: there is no transaction in progress
2026-08-11 00:27:46.196 UTC [15080] WARNING: there is no transaction in progress
2026-08-11 00:27:46.209 UTC [15080] WARNING: there is no transaction in progress
2026-08-11 00:27:46.225 UTC [15080] WARNING: there is no transaction in progress
2026-08-11 00:27:46.236 UTC [15080] WARNING: there is no transaction in progress
2026-08-11 00:27:46.243 UTC [15080] WARNING: there is no transaction in progress
2026-08-11 00:27:46.250 UTC [15080] WARNING: there is no transaction in progress
2026-08-11 00:27:46.258 UTC [15080] WARNING: there is no transaction in progress
2026-08-11 00:27:47,085 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 00:27:47,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 00:27:47,357 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 00:27:47,358 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 00:27:47,662 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 00:27:47,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 00:27:47,954 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 00:27:47,954 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 00:27:48,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 00:27:48,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-fd7l.c.gcp-upstream.internal';"
2026-08-11 00:27:48,798 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 00:27:48,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 00:27:49,109 [I] DB_PATCHES have already been applied
I20260811 00:27:49.115927 2528 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ranger-admin:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskY6MNab/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 00:27:49.116034 2528 mini_ranger.cc:193] Using host: 127.2.120.20
I20260811 00:27:49.137305 2528 mini_ranger.cc:227] Ranger admin URL: http://127.2.120.20:57707
Aug 11, 2026 12:27:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:27:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:27:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 12:27:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 12:27:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 12:27:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 12:27:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 00:27:51.747342 2528 mini_ranger.cc:160] Time spent starting Ranger: real 11.543s user 0.003s sys 0.074s
Aug 11, 2026 12:28:13 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 12:28:13 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 12:28:13 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 12:28:13 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 12:28:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 12:28:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 12:28:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 12:28:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 12:28:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 12:28:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 12:28:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 12:28:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 12:28:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 12:28:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 12:28:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 12:28:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 12:28:13 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 12:28:14 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 00:28:15.347535 2528 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.120.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.120.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.120.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/rangerkms_127.2.120.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.120.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/rangerkms_127.2.120.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.120.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.120.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.120.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/HTTP_127.2.120.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.120.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/HTTP_127.2.120.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 00:28:15 dist-test-slave-fd7l krb5kdc[14831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408095, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 00:28:15.597636 2528 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 00:28:15.632325 2528 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 00:28:16.732679 2528 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 00:28:16.732880 2528 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ranger-kms
I20260811 00:28:16.733045 2528 mini_ranger_kms.cc:227] Using postgres at 127.2.120.20:56291
2026-08-11 00:28:17,656 [I] DB FLAVOR :POSTGRES
2026-08-11 00:28:17,657 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 00:28:17,657 [I] Checking connection
2026-08-11 00:28:17,657 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 00:28:17,920 [I] connection success
2026-08-11 00:28:17,920 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 00:28:17,920 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 00:28:17,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 00:28:18,197 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 00:28:18,197 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 00:28:18,197 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 00:28:18,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:56291/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 00:28:18,520 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 00:28:19.127188 2528 mini_ranger_kms.cc:310] Created kms service
I20260811 00:28:19.348061 2528 mini_ranger_kms.cc:326] Created kudu user
I20260811 00:28:19.399315 2528 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 00:28:19 dist-test-slave-fd7l krb5kdc[14831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408095, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
I20260811 00:28:19.907732 2528 mini_ranger_kms.cc:384] Added ranger policy
I20260811 00:28:19.909585 2528 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ranger-kms:/tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 00:28:19.909686 2528 mini_ranger_kms.cc:240] Using host: 127.2.120.20
I20260811 00:28:19.936647 2528 mini_ranger_kms.cc:276] Ranger KMS PID: 15301
I20260811 00:28:19.936889 2528 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.120.20:46371
Aug 11, 2026 12:28:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:28:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:28:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 12:28:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 12:28:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 12:28:21 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 00:28:22.492810 2528 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.895s user 0.200s sys 0.458s
I20260811 00:28:22.493201 2528 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 00:28:22.493486 2528 mini_ranger_kms.cc:401] 127.2.120.20:46371/kms/v1/keys
Aug 11 00:28:22 dist-test-slave-fd7l krb5kdc[14831](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786408102, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.120.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 00:28:23 dist-test-slave-fd7l krb5kdc[14831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408102, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.120.20@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
Aug 11 00:28:23 dist-test-slave-fd7l krb5kdc[14831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408102, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.120.20@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
Aug 11 00:28:26 dist-test-slave-fd7l krb5kdc[14831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:28:26.326566 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:40565
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:32995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:40565
--ranger_config_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.120.20:46371/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:28:26.912861 15354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:28:26.913249 15354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:28:26.913491 15354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:28:26.924364 15354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:28:26.924669 15354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:28:26.924795 15354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:28:26.924917 15354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:28:26.925060 15354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:28:26.941006 15354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:32995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.120.20:46371/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:40565
--ranger_config_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:40565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:28:26.942672 15354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:28:26.944413 15354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:28:26.966269 15359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:28:26.966491 15361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:28:26.967991 15363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:28:26.968854 15354 server_base.cc:1034] running on GCE node
I20260811 00:28:26.970105 15354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:28:26.974043 15354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:28:26.975445 15354 hybrid_clock.cc:648] HybridClock initialized: now 1786408106975316 us; error 54 us; skew 500 ppm
I20260811 00:28:26.976049 15354 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:28:26 dist-test-slave-fd7l krb5kdc[14831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408106, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:28:26.986711 15354 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:28:26.989848 15354 webserver.cc:460] Webserver started at http://127.2.120.62:33583/ using document root <none> and password file <none>
I20260811 00:28:26.991194 15354 fs_manager.cc:359] Metadata directory not provided
I20260811 00:28:26.991474 15354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:28:26.992019 15354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:28:26 dist-test-slave-fd7l krb5kdc[14831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408106, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
I20260811 00:28:27.068961 15354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "bed6f2d4a17f4f5989bb3ed17e128a14"
format_stamp: "Formatted at 2026-08-11 00:28:27 on dist-test-slave-fd7l"
server_key: "b805a653b28a7edeacacfdfbdd572594"
server_key_iv: "4669509a42cd7d44715c9510fa82fdb0"
server_key_version: "kuduclusterkey@0"
I20260811 00:28:27.070734 15354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "bed6f2d4a17f4f5989bb3ed17e128a14"
format_stamp: "Formatted at 2026-08-11 00:28:27 on dist-test-slave-fd7l"
server_key: "b805a653b28a7edeacacfdfbdd572594"
server_key_iv: "4669509a42cd7d44715c9510fa82fdb0"
server_key_version: "kuduclusterkey@0"
I20260811 00:28:27.082031 15354 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.010s sys 0.002s
I20260811 00:28:27.108428 15372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:28:27.110922 15354 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 00:28:27.111297 15354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "bed6f2d4a17f4f5989bb3ed17e128a14"
format_stamp: "Formatted at 2026-08-11 00:28:27 on dist-test-slave-fd7l"
server_key: "b805a653b28a7edeacacfdfbdd572594"
server_key_iv: "4669509a42cd7d44715c9510fa82fdb0"
server_key_version: "kuduclusterkey@0"
I20260811 00:28:27.111749 15354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:28:27.142983 15354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:28:27.154951 15354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:28:27.155805 15354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:28:27.214179 15354 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:40565
I20260811 00:28:27.214133 15429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:40565 every 8 connection(s)
I20260811 00:28:27.216704 15354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:28:27.222373 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 15354
I20260811 00:28:27.222661 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:28:27.223783 2528 external_mini_cluster.cc:1428] Setting key 922f8c7998a054f48686d7d1f77d0fbe
I20260811 00:28:27.224319 15430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:28:27.248610 15430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14: Bootstrap starting.
I20260811 00:28:27.253487 15430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14: Neither blocks nor log segments found. Creating new log.
I20260811 00:28:27.255929 15430 log.cc:824] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:28:27 dist-test-slave-fd7l krb5kdc[14831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:28:27.263240 15430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14: No bootstrap required, opened a new log
I20260811 00:28:27.276691 15430 raft_consensus.cc:348] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bed6f2d4a17f4f5989bb3ed17e128a14" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40565 } }
I20260811 00:28:27.277120 15434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:27.226472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35474 (local address 127.2.120.62:40565)
0811 00:28:27.241396 (+ 14924us) server_negotiation.cc:206] Beginning negotiation
0811 00:28:27.241418 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 00:28:27.241524 (+ 106us) server_negotiation.cc:407] Connection header received
0811 00:28:27.248666 (+ 7142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:28:27.248704 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:28:27.249342 (+ 638us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:28:27.249838 (+ 496us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:28:27.251538 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:27.254614 (+ 3076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:27.256752 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:27.258369 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:27.258673 (+ 304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:27.264357 (+ 5684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:28:27.264371 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:28:27.264400 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:28:27.264466 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:28:27.270203 (+ 5737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:28:27.271520 (+ 1317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:28:27.271531 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:28:27.271543 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:28:27.271846 (+ 303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:28:27.272492 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:28:27.272497 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:28:27.272500 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:28:27.273681 (+ 1181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:28:27.274068 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0811 00:28:27.274650 (+ 582us) server_negotiation.cc:299] Negotiation successful
0811 00:28:27.275211 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14530,"thread_start_us":14938,"threads_started":1}
I20260811 00:28:27.277241 15430 raft_consensus.cc:374] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:28:27.277495 15430 raft_consensus.cc:729] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bed6f2d4a17f4f5989bb3ed17e128a14, State: Initialized, Role: FOLLOWER
I20260811 00:28:27.278455 15430 consensus_queue.cc:260] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bed6f2d4a17f4f5989bb3ed17e128a14" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40565 } }
I20260811 00:28:27.278800 15430 raft_consensus.cc:388] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:28:27.279078 15430 raft_consensus.cc:482] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:28:27.279462 15430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:28:27.283362 15430 raft_consensus.cc:504] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bed6f2d4a17f4f5989bb3ed17e128a14" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40565 } }
I20260811 00:28:27.284152 15430 leader_election.cc:302] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bed6f2d4a17f4f5989bb3ed17e128a14; no voters:
I20260811 00:28:27.285378 15430 leader_election.cc:288] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:28:27.285529 15436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:28:27.288745 15430 sys_catalog.cc:564] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:28:27.293731 15436 raft_consensus.cc:686] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14 [term 1 LEADER]: Becoming Leader. State: Replica: bed6f2d4a17f4f5989bb3ed17e128a14, State: Running, Role: LEADER
I20260811 00:28:27.294605 15436 consensus_queue.cc:237] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bed6f2d4a17f4f5989bb3ed17e128a14" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40565 } }
I20260811 00:28:27.301215 15437 sys_catalog.cc:455] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bed6f2d4a17f4f5989bb3ed17e128a14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bed6f2d4a17f4f5989bb3ed17e128a14" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40565 } } }
I20260811 00:28:27.301923 15437 sys_catalog.cc:458] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14 [sys.catalog]: This master's current role is: LEADER
I20260811 00:28:27.305069 15438 sys_catalog.cc:455] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bed6f2d4a17f4f5989bb3ed17e128a14. Latest consensus state: current_term: 1 leader_uuid: "bed6f2d4a17f4f5989bb3ed17e128a14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bed6f2d4a17f4f5989bb3ed17e128a14" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40565 } } }
I20260811 00:28:27.305759 15438 sys_catalog.cc:458] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14 [sys.catalog]: This master's current role is: LEADER
I20260811 00:28:27.312350 15445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:28:27.339887 15445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:28:27.369158 15445 catalog_manager.cc:1347] Generated new cluster ID: 6699bb92e9914e2ab779d1e5614ba9c5
I20260811 00:28:27.369417 15445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:28:27.395911 15445 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:28:27.397258 15430 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 00:28:27.398613 15445 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:28:27.431210 15445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14: Generated new TSK 0
I20260811 00:28:27.432900 15445 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
I20260811 00:28:28.410189 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.120.20:46371/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:40565
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:32995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:28:28 dist-test-slave-fd7l krb5kdc[14831](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786408108, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 00:28:29.116756 15489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:28:29.117292 15489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:28:29.117595 15489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:28:29.133888 15489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:28:29.134269 15489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:28:29.134737 15489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:28:29.154150 15489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:32995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.120.20:46371/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:40565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:28:29.155912 15489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:28:29.158406 15489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:28:29.182518 15499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:28:29.182463 15500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:28:29.182814 15502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:28:29.185339 15489 server_base.cc:1034] running on GCE node
I20260811 00:28:29.186134 15489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:28:29.187455 15489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:28:29.190976 15489 hybrid_clock.cc:648] HybridClock initialized: now 1786408109190805 us; error 106 us; skew 500 ppm
I20260811 00:28:29.191596 15489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:28:29 dist-test-slave-fd7l krb5kdc[14831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408109, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:28:29.196698 15489 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:28:29.201045 15489 webserver.cc:460] Webserver started at http://127.2.120.1:33611/ using document root <none> and password file <none>
I20260811 00:28:29.203150 15489 fs_manager.cc:359] Metadata directory not provided
I20260811 00:28:29.203416 15489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:28:29.204188 15489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:28:29 dist-test-slave-fd7l krb5kdc[14831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408109, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
I20260811 00:28:29.239161 15489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "0e3fe0dbcdf54b498c9f5fb9732d04ec"
format_stamp: "Formatted at 2026-08-11 00:28:29 on dist-test-slave-fd7l"
server_key: "58e5e500176073b1b9c2b793f3e91e13"
server_key_iv: "f553a7c2366b8476192cd5f4a391382e"
server_key_version: "kuduclusterkey@0"
I20260811 00:28:29.240924 15489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "0e3fe0dbcdf54b498c9f5fb9732d04ec"
format_stamp: "Formatted at 2026-08-11 00:28:29 on dist-test-slave-fd7l"
server_key: "58e5e500176073b1b9c2b793f3e91e13"
server_key_iv: "f553a7c2366b8476192cd5f4a391382e"
server_key_version: "kuduclusterkey@0"
I20260811 00:28:29.258272 15489 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.009s sys 0.005s
I20260811 00:28:29.290547 15509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:28:29.293175 15489 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.005s sys 0.007s
I20260811 00:28:29.293504 15489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "0e3fe0dbcdf54b498c9f5fb9732d04ec"
format_stamp: "Formatted at 2026-08-11 00:28:29 on dist-test-slave-fd7l"
server_key: "58e5e500176073b1b9c2b793f3e91e13"
server_key_iv: "f553a7c2366b8476192cd5f4a391382e"
server_key_version: "kuduclusterkey@0"
I20260811 00:28:29.293890 15489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:28:29.322851 15489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:28:29.331399 15489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:28:29.332006 15489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:28:29.334293 15489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:28:29.338112 15489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:28:29.338418 15489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 00:28:29.338831 15489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:28:29.339107 15489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:28:29 dist-test-slave-fd7l krb5kdc[14831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408108, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
I20260811 00:28:29.627532 15489 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:33075
I20260811 00:28:29.627542 15639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:33075 every 8 connection(s)
I20260811 00:28:29.629884 15489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:28:29.637094 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 15489
I20260811 00:28:29.637480 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:28:29.638837 2528 external_mini_cluster.cc:1428] Setting key 72cfcf2a3d4a599b93e89db9d9c33439
Aug 11 00:28:29 dist-test-slave-fd7l krb5kdc[14831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408109, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:28:29.682166 15644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:29.635742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:40339 (local address 127.2.120.62:40565)
0811 00:28:29.636728 (+ 986us) server_negotiation.cc:206] Beginning negotiation
0811 00:28:29.636739 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:28:29.641024 (+ 4285us) server_negotiation.cc:407] Connection header received
0811 00:28:29.647651 (+ 6627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:28:29.647662 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:28:29.647805 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:28:29.647964 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:28:29.651996 (+ 4032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:29.653770 (+ 1774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:29.657964 (+ 4194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:29.658567 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:29.658811 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:29.667085 (+ 8274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:28:29.667099 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:28:29.667108 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:28:29.667256 (+ 148us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:28:29.674310 (+ 7054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:28:29.676175 (+ 1865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:28:29.676183 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:28:29.676187 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:28:29.676442 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:28:29.678182 (+ 1740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:28:29.678191 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:28:29.678194 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:28:29.679028 (+ 834us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:28:29.679240 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 00:28:29.681011 (+ 1771us) server_negotiation.cc:299] Negotiation successful
0811 00:28:29.681417 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":1060,"threads_started":1}
WARNING: no policy specified for kudu/127.2.120.2@KRBTEST.COM; defaulting to no policy
I20260811 00:28:29.684404 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:29.637467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:40565 (local address 127.2.120.1:40339)
0811 00:28:29.640696 (+ 3229us) negotiation.cc:107] Waiting for socket to connect
0811 00:28:29.640821 (+ 125us) client_negotiation.cc:174] Beginning negotiation
0811 00:28:29.646968 (+ 6147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:28:29.648482 (+ 1514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:28:29.648500 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:28:29.649682 (+ 1182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:28:29.651477 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:28:29.651593 (+ 116us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:28:29.654249 (+ 2656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:28:29.654260 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:28:29.657587 (+ 3327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:28:29.657625 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:28:29.659058 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:28:29.659072 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:28:29.659449 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:29.660832 (+ 1383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:28:29.660885 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:28:29.666340 (+ 5455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:28:29.674723 (+ 8383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:28:29.674738 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:28:29.674779 (+ 41us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:28:29.675925 (+ 1146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:28:29.676712 (+ 787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:28:29.676728 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:28:29.676732 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:28:29.677895 (+ 1163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:28:29.679286 (+ 1391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:28:29.679298 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:28:29.680042 (+ 744us) client_negotiation.cc:769] Sending connection context
0811 00:28:29.680848 (+ 806us) client_negotiation.cc:240] Negotiation successful
0811 00:28:29.681925 (+ 1077us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1762,"thread_start_us":1178,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.2@KRBTEST.COM" created.
I20260811 00:28:29.687253 15641 heartbeater.cc:347] Connected to a master server at 127.2.120.62:40565
I20260811 00:28:29.687667 15641 heartbeater.cc:464] Registering TS with master...
I20260811 00:28:29.689005 15641 heartbeater.cc:511] Master 127.2.120.62:40565 requested a full tablet report, sending...
I20260811 00:28:29.692742 15390 ts_manager.cc:194] Registered new tserver with Master: 0e3fe0dbcdf54b498c9f5fb9732d04ec (127.2.120.1:33075)
I20260811 00:28:29.695302 15390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:40339
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
I20260811 00:28:29.733361 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.120.20:46371/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:40565
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:32995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:28:29 dist-test-slave-fd7l krb5kdc[14831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408108, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
W20260811 00:28:30.556082 15649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:28:30.556465 15649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:28:30.556751 15649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:28:30.569244 15649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:28:30.569509 15649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:28:30.569798 15649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:28:30.583501 15649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:32995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.120.20:46371/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:40565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:28:30.585208 15649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:28:30.586695 15649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:28:30.609360 15664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:28:30.609539 15665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:28:30.609540 15668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:28:30.614899 15649 server_base.cc:1034] running on GCE node
I20260811 00:28:30.615860 15649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:28:30.617744 15649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:28:30.619045 15649 hybrid_clock.cc:648] HybridClock initialized: now 1786408110618920 us; error 86 us; skew 500 ppm
I20260811 00:28:30.619685 15649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:28:30 dist-test-slave-fd7l krb5kdc[14831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408110, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:28:30.624073 15649 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:28:30.626209 15649 webserver.cc:460] Webserver started at http://127.2.120.2:46093/ using document root <none> and password file <none>
I20260811 00:28:30.627156 15649 fs_manager.cc:359] Metadata directory not provided
I20260811 00:28:30.627316 15649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:28:30.627691 15649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:28:30 dist-test-slave-fd7l krb5kdc[14831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408110, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
I20260811 00:28:30.656594 15649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "5fafe3b60f0e418989fb2901faa72ca7"
format_stamp: "Formatted at 2026-08-11 00:28:30 on dist-test-slave-fd7l"
server_key: "422cf3bdf78835f5e32de0c7a34ce6cc"
server_key_iv: "828adc99e0e7afbc5f141da022d3f831"
server_key_version: "kuduclusterkey@0"
I20260811 00:28:30.658066 15649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "5fafe3b60f0e418989fb2901faa72ca7"
format_stamp: "Formatted at 2026-08-11 00:28:30 on dist-test-slave-fd7l"
server_key: "422cf3bdf78835f5e32de0c7a34ce6cc"
server_key_iv: "828adc99e0e7afbc5f141da022d3f831"
server_key_version: "kuduclusterkey@0"
I20260811 00:28:30.668344 15649 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.009s sys 0.001s
I20260811 00:28:30.693508 15675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:28:30.695768 15649 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260811 00:28:30.696121 15649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "5fafe3b60f0e418989fb2901faa72ca7"
format_stamp: "Formatted at 2026-08-11 00:28:30 on dist-test-slave-fd7l"
server_key: "422cf3bdf78835f5e32de0c7a34ce6cc"
server_key_iv: "828adc99e0e7afbc5f141da022d3f831"
server_key_version: "kuduclusterkey@0"
I20260811 00:28:30.696597 15649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:28:30.700826 15641 heartbeater.cc:503] Master 127.2.120.62:40565 was elected leader, sending a full tablet report...
I20260811 00:28:30.740172 15649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:28:30.747481 15649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:28:30.748113 15649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:28:30.749953 15649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:28:30.753785 15649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:28:30.753933 15649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:28:30.754083 15649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:28:30.754159 15649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:28:30.856739 15649 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:46719
I20260811 00:28:30.856961 15811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:46719 every 8 connection(s)
I20260811 00:28:30.859031 15649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:28:30.865746 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 15649
I20260811 00:28:30.866029 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:28:30.866912 2528 external_mini_cluster.cc:1428] Setting key 6806d997dda21fdfc907caed8966cce6
Aug 11 00:28:30 dist-test-slave-fd7l krb5kdc[14831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408110, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:28:30.896484 15815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:30.864438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:45991 (local address 127.2.120.62:40565)
0811 00:28:30.868669 (+ 4231us) server_negotiation.cc:206] Beginning negotiation
0811 00:28:30.868677 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:28:30.870249 (+ 1572us) server_negotiation.cc:407] Connection header received
0811 00:28:30.875249 (+ 5000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:28:30.875257 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:28:30.875381 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:28:30.875547 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:28:30.878959 (+ 3412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:30.880781 (+ 1822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:30.884214 (+ 3433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:30.884748 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:30.884928 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:30.889113 (+ 4185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:28:30.889122 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:28:30.889127 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:28:30.889191 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:28:30.892429 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:28:30.893607 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:28:30.893615 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:28:30.893624 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:28:30.893783 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:28:30.894344 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:28:30.894349 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:28:30.894351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:28:30.894687 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:28:30.894929 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 00:28:30.895756 (+ 827us) server_negotiation.cc:299] Negotiation successful
0811 00:28:30.896050 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3980,"thread_start_us":3843,"threads_started":1}
I20260811 00:28:30.897787 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:30.865406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:40565 (local address 127.2.120.2:45991)
0811 00:28:30.870030 (+ 4624us) negotiation.cc:107] Waiting for socket to connect
0811 00:28:30.870126 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0811 00:28:30.874692 (+ 4566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:28:30.875982 (+ 1290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:28:30.876000 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:28:30.877248 (+ 1248us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:28:30.878581 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:28:30.878621 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:28:30.881099 (+ 2478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:28:30.881108 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:28:30.883954 (+ 2846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:28:30.883976 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:28:30.885091 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:28:30.885100 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:28:30.885380 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:30.886211 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:28:30.886245 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:28:30.888682 (+ 2437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:28:30.892691 (+ 4009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:28:30.892705 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:28:30.892736 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:28:30.893432 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:28:30.893932 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:28:30.893938 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:28:30.893941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:28:30.894193 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:28:30.894868 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:28:30.894886 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:28:30.895314 (+ 428us) client_negotiation.cc:769] Sending connection context
0811 00:28:30.895647 (+ 333us) client_negotiation.cc:240] Negotiation successful
0811 00:28:30.896263 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4127,"thread_start_us":3559,"threads_started":1}
I20260811 00:28:30.899583 15812 heartbeater.cc:347] Connected to a master server at 127.2.120.62:40565
I20260811 00:28:30.900098 15812 heartbeater.cc:464] Registering TS with master...
I20260811 00:28:30.901361 15812 heartbeater.cc:511] Master 127.2.120.62:40565 requested a full tablet report, sending...
I20260811 00:28:30.903718 15390 ts_manager.cc:194] Registered new tserver with Master: 5fafe3b60f0e418989fb2901faa72ca7 (127.2.120.2:46719)
I20260811 00:28:30.904938 15390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:45991
WARNING: no policy specified for kudu/127.2.120.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
I20260811 00:28:30.954190 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.120.20:46371/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:40565
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:32995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:28:31.607995 15820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:28:31.608346 15820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:28:31.608559 15820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:28:31.618914 15820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:28:31.619180 15820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:28:31.619385 15820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:28:31.635003 15820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:32995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.120.20:46371/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:40565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:28:31.637208 15820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:28:31.638914 15820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:28:31.659322 15831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:28:31.659435 15827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:28:31.659232 15826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:28:31.662488 15820 server_base.cc:1034] running on GCE node
I20260811 00:28:31.664275 15820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:28:31.666823 15820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:28:31.668874 15820 hybrid_clock.cc:648] HybridClock initialized: now 1786408111668715 us; error 163 us; skew 500 ppm
I20260811 00:28:31.669809 15820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:28:31 dist-test-slave-fd7l krb5kdc[14831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408111, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:28:31.677791 15820 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:28:31.681979 15820 webserver.cc:460] Webserver started at http://127.2.120.3:46803/ using document root <none> and password file <none>
I20260811 00:28:31.684144 15820 fs_manager.cc:359] Metadata directory not provided
I20260811 00:28:31.684707 15820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:28:31.685599 15820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:28:31 dist-test-slave-fd7l krb5kdc[14831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408111, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
I20260811 00:28:31.713402 15820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "249cc42c3d2d47bebaf3373be741b1bf"
format_stamp: "Formatted at 2026-08-11 00:28:31 on dist-test-slave-fd7l"
server_key: "f17d9aed0a24f17b679008c4a1c8d03d"
server_key_iv: "dffec25ebc5c251343824646b38c2fc6"
server_key_version: "kuduclusterkey@0"
I20260811 00:28:31.715032 15820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "249cc42c3d2d47bebaf3373be741b1bf"
format_stamp: "Formatted at 2026-08-11 00:28:31 on dist-test-slave-fd7l"
server_key: "f17d9aed0a24f17b679008c4a1c8d03d"
server_key_iv: "dffec25ebc5c251343824646b38c2fc6"
server_key_version: "kuduclusterkey@0"
I20260811 00:28:31.726343 15820 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.005s sys 0.006s
I20260811 00:28:31.749745 15838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:28:31.752403 15820 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260811 00:28:31.752772 15820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "249cc42c3d2d47bebaf3373be741b1bf"
format_stamp: "Formatted at 2026-08-11 00:28:31 on dist-test-slave-fd7l"
server_key: "f17d9aed0a24f17b679008c4a1c8d03d"
server_key_iv: "dffec25ebc5c251343824646b38c2fc6"
server_key_version: "kuduclusterkey@0"
I20260811 00:28:31.753075 15820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:28:31.784912 15820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:28:31.791890 15820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:28:31.792402 15820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:28:31.794323 15820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:28:31.797565 15820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:28:31.797788 15820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:28:31.798045 15820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:28:31.798259 15820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:28:31.877035 15820 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:46291
I20260811 00:28:31.877027 15964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:46291 every 8 connection(s)
I20260811 00:28:31.878846 15820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:28:31.889117 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 15820
I20260811 00:28:31.889396 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:28:31.890087 2528 external_mini_cluster.cc:1428] Setting key db57b0c7200edb514dba22ee8be2fa17
Aug 11 00:28:31 dist-test-slave-fd7l krb5kdc[14831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408111, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:28:31.908429 15812 heartbeater.cc:503] Master 127.2.120.62:40565 was elected leader, sending a full tablet report...
I20260811 00:28:31.908684 15969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:31.882284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:44565 (local address 127.2.120.62:40565)
0811 00:28:31.883085 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0811 00:28:31.883090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:28:31.884498 (+ 1408us) server_negotiation.cc:407] Connection header received
0811 00:28:31.887050 (+ 2552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:28:31.887056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:28:31.887202 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:28:31.887339 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:28:31.889654 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:31.890856 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:31.893842 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:31.894403 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:31.894584 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:31.899926 (+ 5342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:28:31.899933 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:28:31.899936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:28:31.899986 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:28:31.903666 (+ 3680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:28:31.904798 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:28:31.904804 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:28:31.904806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:28:31.904983 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:28:31.905680 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:28:31.905685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:28:31.905687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:28:31.906412 (+ 725us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:28:31.906683 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 00:28:31.907807 (+ 1124us) server_negotiation.cc:299] Negotiation successful
0811 00:28:31.908094 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":506,"threads_started":1}
I20260811 00:28:31.910045 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:31.882780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:40565 (local address 127.2.120.3:44565)
0811 00:28:31.884292 (+ 1512us) negotiation.cc:107] Waiting for socket to connect
0811 00:28:31.884345 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 00:28:31.886695 (+ 2350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:28:31.887633 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:28:31.887644 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:28:31.888328 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:28:31.889358 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:28:31.889400 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:28:31.891092 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:28:31.891098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:28:31.893624 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:28:31.893645 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:28:31.894807 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:28:31.894827 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:28:31.895281 (+ 454us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:31.896291 (+ 1010us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:28:31.896332 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:28:31.899487 (+ 3155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:28:31.903905 (+ 4418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:28:31.903916 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:28:31.903936 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:28:31.904500 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:28:31.905129 (+ 629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:28:31.905134 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:28:31.905138 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:28:31.905463 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:28:31.906683 (+ 1220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:28:31.906693 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:28:31.907272 (+ 579us) client_negotiation.cc:769] Sending connection context
0811 00:28:31.907662 (+ 390us) client_negotiation.cc:240] Negotiation successful
0811 00:28:31.908268 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1239,"thread_start_us":895,"threads_started":1}
I20260811 00:28:31.912065 15967 heartbeater.cc:347] Connected to a master server at 127.2.120.62:40565
I20260811 00:28:31.912448 15967 heartbeater.cc:464] Registering TS with master...
I20260811 00:28:31.914188 15967 heartbeater.cc:511] Master 127.2.120.62:40565 requested a full tablet report, sending...
I20260811 00:28:31.916752 15390 ts_manager.cc:194] Registered new tserver with Master: 249cc42c3d2d47bebaf3373be741b1bf (127.2.120.3:46291)
I20260811 00:28:31.917958 15390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:44565
I20260811 00:28:31.922688 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:28:31.947139 15969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:31.929142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35476 (local address 127.2.120.62:40565)
0811 00:28:31.929489 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0811 00:28:31.929498 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:28:31.929969 (+ 471us) server_negotiation.cc:407] Connection header received
0811 00:28:31.930148 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:28:31.930152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:28:31.930254 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:28:31.930374 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:28:31.932162 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:31.933084 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:31.935067 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:31.935836 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:31.936086 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:31.939700 (+ 3614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:28:31.939706 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:28:31.939710 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:28:31.939757 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:28:31.943377 (+ 3620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:28:31.944453 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:28:31.944458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:28:31.944461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:28:31.944676 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:28:31.945176 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:28:31.945181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:28:31.945183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:28:31.945546 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:28:31.945789 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 00:28:31.946594 (+ 805us) server_negotiation.cc:299] Negotiation successful
0811 00:28:31.946847 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 00:28:31.954684 15390 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:28:31.968482 15390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:28:32.012050 15984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:31.990696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60968 (local address 127.2.120.1:33075)
0811 00:28:31.998428 (+ 7732us) server_negotiation.cc:206] Beginning negotiation
0811 00:28:31.998443 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:28:31.998498 (+ 55us) server_negotiation.cc:407] Connection header received
0811 00:28:31.998698 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:28:31.998711 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:28:31.999106 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:28:32.001111 (+ 2005us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:28:32.001645 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.003729 (+ 2084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.008071 (+ 4342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.010101 (+ 2030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.010307 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:32.010626 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 00:28:32.011077 (+ 451us) server_negotiation.cc:299] Negotiation successful
0811 00:28:32.011272 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7479,"thread_start_us":7247,"threads_started":1}
I20260811 00:28:32.012496 15983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:31.989562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:33075 (local address 127.0.0.1:60968)
0811 00:28:31.992156 (+ 2594us) negotiation.cc:107] Waiting for socket to connect
0811 00:28:31.992183 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 00:28:31.992356 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:28:31.999580 (+ 7224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:28:31.999594 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:28:31.999651 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:28:32.000743 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:28:32.000768 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.003911 (+ 3143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.003920 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:28:32.007775 (+ 3855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:28:32.007791 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.010523 (+ 2732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.010533 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:28:32.010792 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:32.010833 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 00:28:32.011041 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 00:28:32.011273 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2405,"thread_start_us":2135,"threads_started":1}
I20260811 00:28:32.015782 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:31.989151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34204 (local address 127.2.120.2:46719)
0811 00:28:32.000449 (+ 11298us) server_negotiation.cc:206] Beginning negotiation
0811 00:28:32.000463 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:28:32.001764 (+ 1301us) server_negotiation.cc:407] Connection header received
0811 00:28:32.002698 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:28:32.002707 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:28:32.003123 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:28:32.004303 (+ 1180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:28:32.005312 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.007709 (+ 2397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.011651 (+ 3942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.013814 (+ 2163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.014046 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:32.014496 (+ 450us) server_negotiation.cc:1035] Waiting for connection context
0811 00:28:32.014746 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 00:28:32.015034 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10958,"thread_start_us":12473,"threads_started":1}
I20260811 00:28:32.016077 15981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:31.988635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:46719 (local address 127.0.0.1:34204)
0811 00:28:31.990488 (+ 1853us) negotiation.cc:107] Waiting for socket to connect
0811 00:28:31.990570 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0811 00:28:31.990899 (+ 329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:28:32.003538 (+ 12639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:28:32.003544 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:28:32.003567 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:28:32.005050 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:28:32.005069 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.008038 (+ 2969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.008045 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:28:32.011389 (+ 3344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:28:32.011407 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.014234 (+ 2827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.014242 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:28:32.014522 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:32.014560 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 00:28:32.014677 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 00:28:32.014826 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1189,"mutex_wait_us":23,"thread_start_us":959,"threads_started":1}
I20260811 00:28:32.017853 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:31.992682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:46291 (local address 127.0.0.1:55556)
0811 00:28:32.003213 (+ 10531us) negotiation.cc:107] Waiting for socket to connect
0811 00:28:32.003238 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:28:32.003538 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:28:32.004836 (+ 1298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:28:32.004843 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:28:32.004893 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:28:32.006104 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:28:32.006121 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.008575 (+ 2454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.008585 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:28:32.013106 (+ 4521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:28:32.013122 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.016097 (+ 2975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.016105 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:28:32.016302 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:32.016335 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 00:28:32.016426 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 00:28:32.016611 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10314,"mutex_wait_us":29,"thread_start_us":9928,"threads_started":1}
I20260811 00:28:32.018563 15985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:31.991349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55556 (local address 127.2.120.3:46291)
0811 00:28:32.001699 (+ 10350us) server_negotiation.cc:206] Beginning negotiation
0811 00:28:32.001708 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:28:32.003679 (+ 1971us) server_negotiation.cc:407] Connection header received
0811 00:28:32.003825 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:28:32.003847 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:28:32.004226 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:28:32.004540 (+ 314us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:28:32.006410 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.008356 (+ 1946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.013349 (+ 4993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.015123 (+ 1774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.015269 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:32.015637 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0811 00:28:32.016742 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0811 00:28:32.016954 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10070,"thread_start_us":9773,"threads_started":1}
I20260811 00:28:32.018448 15561 tablet_service.cc:1467] Processing CreateTablet for tablet c0188eac177b4811863f912f6094564d (DEFAULT_TABLE table=test-table [id=2c1114afdea94b3185864b25e82cbbf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:28:32.021148 15561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0188eac177b4811863f912f6094564d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:28:32.021404 15736 tablet_service.cc:1467] Processing CreateTablet for tablet c0188eac177b4811863f912f6094564d (DEFAULT_TABLE table=test-table [id=2c1114afdea94b3185864b25e82cbbf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:28:32.023518 15736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0188eac177b4811863f912f6094564d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:28:32.023759 15891 tablet_service.cc:1467] Processing CreateTablet for tablet c0188eac177b4811863f912f6094564d (DEFAULT_TABLE table=test-table [id=2c1114afdea94b3185864b25e82cbbf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:28:32.025951 15891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0188eac177b4811863f912f6094564d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:28:32.038074 15988 tablet_bootstrap.cc:492] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec: Bootstrap starting.
I20260811 00:28:32.043820 15988 tablet_bootstrap.cc:654] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec: Neither blocks nor log segments found. Creating new log.
I20260811 00:28:32.046340 15988 log.cc:824] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec: Log is configured to *not* fsync() on all Append() calls
I20260811 00:28:32.046756 15989 tablet_bootstrap.cc:492] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7: Bootstrap starting.
I20260811 00:28:32.049394 15990 tablet_bootstrap.cc:492] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf: Bootstrap starting.
I20260811 00:28:32.055238 15990 tablet_bootstrap.cc:654] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf: Neither blocks nor log segments found. Creating new log.
I20260811 00:28:32.055897 15989 tablet_bootstrap.cc:654] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7: Neither blocks nor log segments found. Creating new log.
I20260811 00:28:32.057528 15990 log.cc:824] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf: Log is configured to *not* fsync() on all Append() calls
I20260811 00:28:32.057937 15989 log.cc:824] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7: Log is configured to *not* fsync() on all Append() calls
I20260811 00:28:32.058085 15988 tablet_bootstrap.cc:492] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec: No bootstrap required, opened a new log
I20260811 00:28:32.058502 15988 ts_tablet_manager.cc:1397] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.017s
I20260811 00:28:32.063264 15990 tablet_bootstrap.cc:492] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf: No bootstrap required, opened a new log
I20260811 00:28:32.063953 15990 ts_tablet_manager.cc:1397] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.006s
I20260811 00:28:32.065115 15989 tablet_bootstrap.cc:492] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7: No bootstrap required, opened a new log
I20260811 00:28:32.065573 15989 ts_tablet_manager.cc:1397] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7: Time spent bootstrapping tablet: real 0.019s user 0.012s sys 0.003s
I20260811 00:28:32.075335 15990 raft_consensus.cc:348] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fafe3b60f0e418989fb2901faa72ca7" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46719 } } peers { permanent_uuid: "249cc42c3d2d47bebaf3373be741b1bf" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46291 } } peers { permanent_uuid: "0e3fe0dbcdf54b498c9f5fb9732d04ec" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33075 } }
I20260811 00:28:32.075918 15990 raft_consensus.cc:374] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:28:32.076069 15990 raft_consensus.cc:729] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 249cc42c3d2d47bebaf3373be741b1bf, State: Initialized, Role: FOLLOWER
I20260811 00:28:32.076733 15988 raft_consensus.cc:348] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fafe3b60f0e418989fb2901faa72ca7" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46719 } } peers { permanent_uuid: "249cc42c3d2d47bebaf3373be741b1bf" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46291 } } peers { permanent_uuid: "0e3fe0dbcdf54b498c9f5fb9732d04ec" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33075 } }
I20260811 00:28:32.077056 15990 consensus_queue.cc:260] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fafe3b60f0e418989fb2901faa72ca7" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46719 } } peers { permanent_uuid: "249cc42c3d2d47bebaf3373be741b1bf" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46291 } } peers { permanent_uuid: "0e3fe0dbcdf54b498c9f5fb9732d04ec" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33075 } }
I20260811 00:28:32.077404 15988 raft_consensus.cc:374] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:28:32.077574 15988 raft_consensus.cc:729] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e3fe0dbcdf54b498c9f5fb9732d04ec, State: Initialized, Role: FOLLOWER
I20260811 00:28:32.078538 15988 consensus_queue.cc:260] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fafe3b60f0e418989fb2901faa72ca7" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46719 } } peers { permanent_uuid: "249cc42c3d2d47bebaf3373be741b1bf" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46291 } } peers { permanent_uuid: "0e3fe0dbcdf54b498c9f5fb9732d04ec" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33075 } }
I20260811 00:28:32.079027 15967 heartbeater.cc:503] Master 127.2.120.62:40565 was elected leader, sending a full tablet report...
I20260811 00:28:32.080006 15990 ts_tablet_manager.cc:1428] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf: Time spent starting tablet: real 0.016s user 0.011s sys 0.007s
I20260811 00:28:32.084758 15989 raft_consensus.cc:348] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fafe3b60f0e418989fb2901faa72ca7" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46719 } } peers { permanent_uuid: "249cc42c3d2d47bebaf3373be741b1bf" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46291 } } peers { permanent_uuid: "0e3fe0dbcdf54b498c9f5fb9732d04ec" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33075 } }
I20260811 00:28:32.085579 15989 raft_consensus.cc:374] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:28:32.085762 15989 raft_consensus.cc:729] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fafe3b60f0e418989fb2901faa72ca7, State: Initialized, Role: FOLLOWER
I20260811 00:28:32.086727 15989 consensus_queue.cc:260] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fafe3b60f0e418989fb2901faa72ca7" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46719 } } peers { permanent_uuid: "249cc42c3d2d47bebaf3373be741b1bf" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46291 } } peers { permanent_uuid: "0e3fe0dbcdf54b498c9f5fb9732d04ec" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33075 } }
I20260811 00:28:32.087543 15988 ts_tablet_manager.cc:1428] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec: Time spent starting tablet: real 0.029s user 0.019s sys 0.010s
I20260811 00:28:32.090488 15989 ts_tablet_manager.cc:1428] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7: Time spent starting tablet: real 0.025s user 0.014s sys 0.010s
W20260811 00:28:32.113552 15813 tablet.cc:2369] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:28:32.123636 15995 raft_consensus.cc:482] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:28:32.124091 15995 raft_consensus.cc:504] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fafe3b60f0e418989fb2901faa72ca7" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46719 } } peers { permanent_uuid: "249cc42c3d2d47bebaf3373be741b1bf" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46291 } } peers { permanent_uuid: "0e3fe0dbcdf54b498c9f5fb9732d04ec" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33075 } }
I20260811 00:28:32.126458 15995 leader_election.cc:288] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5fafe3b60f0e418989fb2901faa72ca7 (127.2.120.2:46719), 0e3fe0dbcdf54b498c9f5fb9732d04ec (127.2.120.1:33075)
W20260811 00:28:32.131664 15968 tablet.cc:2369] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:28:32.136260 15643 tablet.cc:2369] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:28:32.137166 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:32.127047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:60991 (local address 127.2.120.2:46719)
0811 00:28:32.127478 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0811 00:28:32.127485 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:28:32.127978 (+ 493us) server_negotiation.cc:407] Connection header received
0811 00:28:32.128255 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:28:32.128270 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:28:32.128383 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:28:32.128782 (+ 399us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:28:32.129601 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.131204 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.134282 (+ 3078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.135712 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.135874 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:32.136007 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 00:28:32.136633 (+ 626us) server_negotiation.cc:299] Negotiation successful
0811 00:28:32.136800 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 00:28:32.137202 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:32.127482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:46719 (local address 127.2.120.3:60991)
0811 00:28:32.127815 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 00:28:32.127848 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 00:28:32.128067 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:28:32.128909 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:28:32.128914 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:28:32.128961 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:28:32.129442 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:28:32.129458 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.131402 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.131408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:28:32.134075 (+ 2667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:28:32.134103 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.136009 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.136016 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:28:32.136321 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:32.136363 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 00:28:32.136545 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 00:28:32.136812 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":22}
I20260811 00:28:32.138864 15984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:32.127152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:55527 (local address 127.2.120.1:33075)
0811 00:28:32.127480 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0811 00:28:32.127486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:28:32.129132 (+ 1646us) server_negotiation.cc:407] Connection header received
0811 00:28:32.129550 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:28:32.129556 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:28:32.129635 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:28:32.129740 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:28:32.130877 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.132319 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.135170 (+ 2851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.136969 (+ 1799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.137184 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:32.137295 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 00:28:32.138240 (+ 945us) server_negotiation.cc:299] Negotiation successful
0811 00:28:32.138440 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260811 00:28:32.138881 15998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:32.127866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:33075 (local address 127.2.120.3:55527)
0811 00:28:32.128975 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0811 00:28:32.128994 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:28:32.129359 (+ 365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:28:32.129790 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:28:32.129804 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:28:32.129841 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:28:32.130628 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:28:32.130643 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.132602 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.132607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:28:32.134975 (+ 2368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:28:32.134988 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.137547 (+ 2559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.137553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:28:32.137947 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:32.137995 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 00:28:32.138165 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 00:28:32.138333 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"mutex_wait_us":54,"thread_start_us":636,"threads_started":1}
I20260811 00:28:32.139495 15757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0188eac177b4811863f912f6094564d" candidate_uuid: "249cc42c3d2d47bebaf3373be741b1bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fafe3b60f0e418989fb2901faa72ca7" is_pre_election: true
I20260811 00:28:32.140810 15757 raft_consensus.cc:2454] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 249cc42c3d2d47bebaf3373be741b1bf in term 0.
I20260811 00:28:32.141000 15584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0188eac177b4811863f912f6094564d" candidate_uuid: "249cc42c3d2d47bebaf3373be741b1bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3fe0dbcdf54b498c9f5fb9732d04ec" is_pre_election: true
I20260811 00:28:32.142117 15584 raft_consensus.cc:2454] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 249cc42c3d2d47bebaf3373be741b1bf in term 0.
I20260811 00:28:32.142537 15842 leader_election.cc:302] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 249cc42c3d2d47bebaf3373be741b1bf, 5fafe3b60f0e418989fb2901faa72ca7; no voters:
I20260811 00:28:32.143123 15995 raft_consensus.cc:2781] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:28:32.143407 15995 raft_consensus.cc:482] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:28:32.143594 15995 raft_consensus.cc:3037] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:28:32.147001 15995 raft_consensus.cc:504] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fafe3b60f0e418989fb2901faa72ca7" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46719 } } peers { permanent_uuid: "249cc42c3d2d47bebaf3373be741b1bf" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46291 } } peers { permanent_uuid: "0e3fe0dbcdf54b498c9f5fb9732d04ec" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33075 } }
I20260811 00:28:32.148151 15995 leader_election.cc:288] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [CANDIDATE]: Term 1 election: Requested vote from peers 5fafe3b60f0e418989fb2901faa72ca7 (127.2.120.2:46719), 0e3fe0dbcdf54b498c9f5fb9732d04ec (127.2.120.1:33075)
I20260811 00:28:32.148841 15584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0188eac177b4811863f912f6094564d" candidate_uuid: "249cc42c3d2d47bebaf3373be741b1bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e3fe0dbcdf54b498c9f5fb9732d04ec"
I20260811 00:28:32.148901 15757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0188eac177b4811863f912f6094564d" candidate_uuid: "249cc42c3d2d47bebaf3373be741b1bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fafe3b60f0e418989fb2901faa72ca7"
I20260811 00:28:32.149174 15584 raft_consensus.cc:3037] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:28:32.149333 15757 raft_consensus.cc:3037] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:28:32.153450 15757 raft_consensus.cc:2454] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 249cc42c3d2d47bebaf3373be741b1bf in term 1.
I20260811 00:28:32.153530 15584 raft_consensus.cc:2454] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 249cc42c3d2d47bebaf3373be741b1bf in term 1.
I20260811 00:28:32.154244 15839 leader_election.cc:302] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e3fe0dbcdf54b498c9f5fb9732d04ec, 249cc42c3d2d47bebaf3373be741b1bf; no voters:
I20260811 00:28:32.154774 15995 raft_consensus.cc:2781] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:28:32.155592 15995 raft_consensus.cc:686] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [term 1 LEADER]: Becoming Leader. State: Replica: 249cc42c3d2d47bebaf3373be741b1bf, State: Running, Role: LEADER
I20260811 00:28:32.156229 15995 consensus_queue.cc:237] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fafe3b60f0e418989fb2901faa72ca7" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46719 } } peers { permanent_uuid: "249cc42c3d2d47bebaf3373be741b1bf" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46291 } } peers { permanent_uuid: "0e3fe0dbcdf54b498c9f5fb9732d04ec" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33075 } }
I20260811 00:28:32.163522 15389 catalog_manager.cc:5495] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 249cc42c3d2d47bebaf3373be741b1bf (127.2.120.3). New cstate: current_term: 1 leader_uuid: "249cc42c3d2d47bebaf3373be741b1bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fafe3b60f0e418989fb2901faa72ca7" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "249cc42c3d2d47bebaf3373be741b1bf" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e3fe0dbcdf54b498c9f5fb9732d04ec" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33075 } health_report { overall_health: UNKNOWN } } }
I20260811 00:28:32.219370 15985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:32.209256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55572 (local address 127.2.120.3:46291)
0811 00:28:32.209619 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0811 00:28:32.209625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:28:32.209654 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:28:32.209801 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:28:32.209805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:28:32.209958 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:28:32.210172 (+ 214us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:28:32.211055 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.213027 (+ 1972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.215280 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.216152 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.216323 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:32.217644 (+ 1321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:28:32.218469 (+ 825us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:28:32.218645 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 00:28:32.218884 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 00:28:32.219004 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 00:28:32.231391 15584 raft_consensus.cc:1261] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec [term 1 FOLLOWER]: Refusing update from remote peer 249cc42c3d2d47bebaf3373be741b1bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:28:32.231446 15757 raft_consensus.cc:1261] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7 [term 1 FOLLOWER]: Refusing update from remote peer 249cc42c3d2d47bebaf3373be741b1bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:28:32.232895 15999 consensus_queue.cc:1035] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fafe3b60f0e418989fb2901faa72ca7" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:28:32.233587 15995 consensus_queue.cc:1035] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e3fe0dbcdf54b498c9f5fb9732d04ec" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:28:32.275487 15390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:28:32.275985 15390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:28:32.281669 15390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bed6f2d4a17f4f5989bb3ed17e128a14: Sending DeleteTablet for 3 replicas of tablet c0188eac177b4811863f912f6094564d
I20260811 00:28:32.283638 15561 tablet_service.cc:1514] Processing DeleteTablet for tablet c0188eac177b4811863f912f6094564d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:28:32 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:60968
I20260811 00:28:32.283708 15891 tablet_service.cc:1514] Processing DeleteTablet for tablet c0188eac177b4811863f912f6094564d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:28:32 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:55556
I20260811 00:28:32.283720 15736 tablet_service.cc:1514] Processing DeleteTablet for tablet c0188eac177b4811863f912f6094564d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:28:32 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:34204
I20260811 00:28:32.287559 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 15489
I20260811 00:28:32.296870 16017 tablet_replica.cc:330] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7: stopping tablet replica
I20260811 00:28:32.297698 16017 raft_consensus.cc:2229] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:28:32.299176 16017 raft_consensus.cc:2258] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:28:32.301806 16016 tablet_replica.cc:330] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf: stopping tablet replica
I20260811 00:28:32.302538 16015 tablet_replica.cc:330] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec: stopping tablet replica
I20260811 00:28:32.302727 16016 raft_consensus.cc:2229] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:28:32.303284 16015 raft_consensus.cc:2229] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:28:32.304069 16016 raft_consensus.cc:2258] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:28:32.304359 16015 raft_consensus.cc:2258] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:28:32.303745 16017 ts_tablet_manager.cc:1905] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:28:32.306850 16016 ts_tablet_manager.cc:1905] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:28:32.307777 16015 ts_tablet_manager.cc:1905] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:28:32.315649 16017 ts_tablet_manager.cc:1918] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:28:32.315990 16017 log.cc:1196] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/c0188eac177b4811863f912f6094564d
I20260811 00:28:32.316900 16017 ts_tablet_manager.cc:1939] T c0188eac177b4811863f912f6094564d P 5fafe3b60f0e418989fb2901faa72ca7: Deleting consensus metadata
I20260811 00:28:32.317744 16016 ts_tablet_manager.cc:1918] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:28:32.318121 16016 log.cc:1196] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/c0188eac177b4811863f912f6094564d
I20260811 00:28:32.319223 16016 ts_tablet_manager.cc:1939] T c0188eac177b4811863f912f6094564d P 249cc42c3d2d47bebaf3373be741b1bf: Deleting consensus metadata
I20260811 00:28:32.319366 16015 ts_tablet_manager.cc:1918] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:28:32.319766 16015 log.cc:1196] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/c0188eac177b4811863f912f6094564d
I20260811 00:28:32.319723 15376 catalog_manager.cc:4841] TS 5fafe3b60f0e418989fb2901faa72ca7 (127.2.120.2:46719): tablet c0188eac177b4811863f912f6094564d (table test-table [id=2c1114afdea94b3185864b25e82cbbf0]) successfully deleted
Aug 11 00:28:32 dist-test-slave-fd7l krb5kdc[14831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:28:32.321590 16015 ts_tablet_manager.cc:1939] T c0188eac177b4811863f912f6094564d P 0e3fe0dbcdf54b498c9f5fb9732d04ec: Deleting consensus metadata
I20260811 00:28:32.322100 15376 catalog_manager.cc:4841] TS 249cc42c3d2d47bebaf3373be741b1bf (127.2.120.3:46291): tablet c0188eac177b4811863f912f6094564d (table test-table [id=2c1114afdea94b3185864b25e82cbbf0]) successfully deleted
I20260811 00:28:32.324223 15373 catalog_manager.cc:4841] TS 0e3fe0dbcdf54b498c9f5fb9732d04ec (127.2.120.1:33075): tablet c0188eac177b4811863f912f6094564d (table test-table [id=2c1114afdea94b3185864b25e82cbbf0]) successfully deleted
I20260811 00:28:32.335086 15984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:32.289551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60976 (local address 127.2.120.1:33075)
0811 00:28:32.289880 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0811 00:28:32.289891 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:28:32.300587 (+ 10696us) server_negotiation.cc:407] Connection header received
0811 00:28:32.300880 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:28:32.300890 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:28:32.301085 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:28:32.301234 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:28:32.305071 (+ 3837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.306525 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.310359 (+ 3834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.311315 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.311465 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:32.323199 (+ 11734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:28:32.323220 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:28:32.323254 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:28:32.323354 (+ 100us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:28:32.329058 (+ 5704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:28:32.330759 (+ 1701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:28:32.330773 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:28:32.330777 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:28:32.331100 (+ 323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:28:32.331775 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:28:32.331791 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:28:32.331796 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:28:32.332852 (+ 1056us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:28:32.333431 (+ 579us) server_negotiation.cc:1035] Waiting for connection context
0811 00:28:32.333974 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 00:28:32.334324 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260811 00:28:32.336851 15633 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:28:32.505820 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 15489
I20260811 00:28:32.529634 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 15649
Aug 11 00:28:32 dist-test-slave-fd7l krb5kdc[14831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.2@KRBTEST.COM
I20260811 00:28:32.551522 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:32.530670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34208 (local address 127.2.120.2:46719)
0811 00:28:32.530894 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 00:28:32.530899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:28:32.530920 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:28:32.531036 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:28:32.531040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:28:32.531125 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:28:32.531231 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:28:32.533384 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.534151 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.536419 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.537152 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.537279 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:32.543771 (+ 6492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:28:32.543795 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:28:32.543814 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:28:32.543864 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:28:32.547602 (+ 3738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:28:32.549090 (+ 1488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:28:32.549100 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:28:32.549102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:28:32.549260 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:28:32.549740 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:28:32.549745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:28:32.549747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:28:32.550168 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:28:32.550525 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0811 00:28:32.550887 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 00:28:32.551145 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 00:28:32.552209 15807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:28:32.712458 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 15649
I20260811 00:28:32.753458 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 15820
Aug 11 00:28:32 dist-test-slave-fd7l krb5kdc[14831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.3@KRBTEST.COM
I20260811 00:28:32.774073 16023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:28:32.754453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55582 (local address 127.2.120.3:46291)
0811 00:28:32.755263 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0811 00:28:32.755268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:28:32.755285 (+ 17us) server_negotiation.cc:407] Connection header received
0811 00:28:32.755350 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:28:32.755354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:28:32.755418 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:28:32.755576 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:28:32.757082 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.758022 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.759445 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:28:32.760020 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:28:32.760168 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:28:32.765999 (+ 5831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:28:32.766021 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:28:32.766044 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:28:32.766105 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:28:32.770463 (+ 4358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:28:32.771470 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:28:32.771483 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:28:32.771486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:28:32.771673 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:28:32.772139 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:28:32.772146 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:28:32.772154 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:28:32.772703 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:28:32.773065 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0811 00:28:32.773414 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 00:28:32.773706 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":588,"threads_started":1}
I20260811 00:28:32.775079 15959 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:28:32.920468 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 15820
I20260811 00:28:32.953068 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 15354
I20260811 00:28:32.954850 15425 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:28:33.073072 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 15354
I20260811 00:28:33.106931 2528 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 00:28:33.454288 2528 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 00:28:33.454664 2528 mini_ranger.cc:66] Stopping Ranger...
I20260811 00:28:33.801465 2528 mini_ranger.cc:68] Stopped Ranger
2026-08-11 00:28:33.801 UTC [14868] LOG: received smart shutdown request
2026-08-11 00:28:33.806 UTC [14868] LOG: background worker "logical replication launcher" (PID 14880) exited with exit code 1
2026-08-11 00:28:33.807 UTC [14875] LOG: shutting down
2026-08-11 00:28:33.994 UTC [14868] LOG: database system is shut down
2026-08-11T00:28:33Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (60028 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:28:34 dist-test-slave-fd7l krb5kdc[16037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:28:34 dist-test-slave-fd7l krb5kdc[16037](info): set up 2 sockets
Aug 11 00:28:34 dist-test-slave-fd7l krb5kdc[16037](info): commencing operation
krb5kdc: starting...
W20260811 00:28:35.398936 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:28:35 dist-test-slave-fd7l krb5kdc[16037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:28:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:28:35Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.120.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.120.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.120.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/rangeradmin_127.2.120.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.120.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/rangeradmin_127.2.120.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.120.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.120.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.120.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/rangerlookup_127.2.120.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.120.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/rangerlookup_127.2.120.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.120.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.120.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.120.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/HTTP_127.2.120.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.120.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/HTTP_127.2.120.20@KRBTEST.COM.keytab.
I20260811 00:28:35.817088 2528 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskY6MNab/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/postgres -l logfile start
2026-08-11 00:28:38.711 UTC [16075] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 00:28:38.711 UTC [16075] LOG: listening on IPv4 address "127.2.120.20", port 43077
2026-08-11 00:28:38.716 UTC [16075] LOG: listening on Unix socket "/tmp/.s.PGSQL.43077"
2026-08-11 00:28:38.741 UTC [16078] LOG: database system was shut down at 2026-08-11 00:28:36 UTC
2026-08-11 00:28:38.745 UTC [16075] LOG: database system is ready to accept connections
I20260811 00:28:39.879122 2528 mini_postgres.cc:96] Postgres bound to 43077
2026-08-11 00:28:39.901 UTC [16086] FATAL: database "slave" does not exist
127.2.120.20:43077 - accepting connections
I20260811 00:28:39.902251 2528 mini_ranger.cc:161] Starting Ranger...
I20260811 00:28:39.935762 2528 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 00:28:40.547101 2528 mini_ranger.cc:87] Created ranger Postgres database
I20260811 00:28:40.547238 2528 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-admin
2026-08-11 00:28:40,853 [I] DB FLAVOR :POSTGRES
2026-08-11 00:28:40,853 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 00:28:40,853 [I] Checking connection..
2026-08-11 00:28:40,853 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 00:28:41,131 [I] Checking connection passed.
2026-08-11 00:28:41,131 [I] --------- Verifying version history table ---------
2026-08-11 00:28:41,132 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 00:28:41,410 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 00:28:41,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 00:28:41,700 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 00:28:41,701 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 00:28:41,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 00:28:41.967 UTC [16162] WARNING: there is no transaction in progress
2026-08-11 00:28:41,975 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 00:28:41,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 00:28:42,232 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 00:28:42,233 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 00:28:42,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 00:28:42,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 00:28:42,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-fd7l.c.gcp-upstream.internal','N') ;"
2026-08-11 00:28:43,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 00:28:43,323 [I] Table x_portal_user does not exist in database ranger
2026-08-11 00:28:43,323 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 00:28:43,573 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 00:28:43,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 00:28:43,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 00:28:44,138 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 00:28:44,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 00:28:45.211 UTC [16284] WARNING: there is no transaction in progress
2026-08-11 00:28:45.230 UTC [16284] WARNING: there is no transaction in progress
2026-08-11 00:28:45.249 UTC [16284] WARNING: there is no transaction in progress
2026-08-11 00:28:45.267 UTC [16284] WARNING: there is no transaction in progress
2026-08-11 00:28:45.286 UTC [16284] WARNING: there is no transaction in progress
2026-08-11 00:28:45.305 UTC [16284] WARNING: there is no transaction in progress
2026-08-11 00:28:45.366 UTC [16284] WARNING: there is no transaction in progress
2026-08-11 00:28:45.374 UTC [16284] WARNING: there is no transaction in progress
2026-08-11 00:28:45.389 UTC [16284] WARNING: there is no transaction in progress
2026-08-11 00:28:45.402 UTC [16284] WARNING: there is no transaction in progress
2026-08-11 00:28:45.414 UTC [16284] WARNING: there is no transaction in progress
2026-08-11 00:28:45.430 UTC [16284] WARNING: there is no transaction in progress
2026-08-11 00:28:45.442 UTC [16284] WARNING: there is no transaction in progress
2026-08-11 00:28:45.450 UTC [16284] WARNING: there is no transaction in progress
2026-08-11 00:28:45.463 UTC [16284] WARNING: there is no transaction in progress
2026-08-11 00:28:45.478 UTC [16284] WARNING: there is no transaction in progress
2026-08-11 00:28:46,220 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 00:28:46,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 00:28:46,503 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 00:28:46,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 00:28:46,786 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 00:28:46,786 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 00:28:47,046 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 00:28:47,047 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 00:28:47,354 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 00:28:47,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-fd7l.c.gcp-upstream.internal';"
2026-08-11 00:28:47,923 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 00:28:47,924 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 00:28:48,174 [I] DB_PATCHES have already been applied
I20260811 00:28:48.181061 2528 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-admin:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskY6MNab/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 00:28:48.181164 2528 mini_ranger.cc:193] Using host: 127.2.120.20
I20260811 00:28:48.201231 2528 mini_ranger.cc:227] Ranger admin URL: http://127.2.120.20:51243
Aug 11, 2026 12:28:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:28:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:28:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 12:28:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 12:28:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 12:28:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 12:28:49 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 00:28:49.499183 2528 mini_ranger.cc:160] Time spent starting Ranger: real 9.597s user 0.000s sys 0.058s
Aug 11, 2026 12:29:14 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 12:29:14 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 12:29:14 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 12:29:14 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 12:29:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 12:29:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 12:29:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 12:29:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 12:29:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 12:29:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 12:29:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 12:29:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 12:29:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 12:29:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 12:29:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 12:29:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 12:29:14 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 12:29:15 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 00:29:16.097615 2528 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.120.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.120.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.120.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/rangerkms_127.2.120.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.120.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/rangerkms_127.2.120.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.120.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.120.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.120.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/HTTP_127.2.120.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.120.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/HTTP_127.2.120.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 00:29:16 dist-test-slave-fd7l krb5kdc[16037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408156, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 00:29:16.320459 2528 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 00:29:16.351652 2528 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 00:29:17.353531 2528 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 00:29:17.353868 2528 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-kms
I20260811 00:29:17.354008 2528 mini_ranger_kms.cc:227] Using postgres at 127.2.120.20:43077
2026-08-11 00:29:17,824 [I] DB FLAVOR :POSTGRES
2026-08-11 00:29:17,824 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 00:29:17,825 [I] Checking connection
2026-08-11 00:29:17,825 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 00:29:18,117 [I] connection success
2026-08-11 00:29:18,117 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 00:29:18,117 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 00:29:18,117 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 00:29:18,406 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 00:29:18,406 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 00:29:18,406 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 00:29:18,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.120.20:43077/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 00:29:18,725 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 00:29:19.421882 2528 mini_ranger_kms.cc:310] Created kms service
I20260811 00:29:19.667760 2528 mini_ranger_kms.cc:326] Created kudu user
I20260811 00:29:19.721480 2528 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 00:29:19 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408156, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
I20260811 00:29:20.166520 2528 mini_ranger_kms.cc:384] Added ranger policy
I20260811 00:29:20.166929 2528 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-kms:/tmp/dist-test-taskY6MNab/build/asan/bin/postgresql.jar:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskY6MNab/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 00:29:20.166987 2528 mini_ranger_kms.cc:240] Using host: 127.2.120.20
I20260811 00:29:20.189602 2528 mini_ranger_kms.cc:276] Ranger KMS PID: 16500
I20260811 00:29:20.189759 2528 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.120.20:53481
Aug 11, 2026 12:29:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:29:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:29:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 12:29:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 12:29:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 12:29:21 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 00:29:22.753928 2528 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.433s user 0.087s sys 0.413s
I20260811 00:29:22.754215 2528 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 00:29:22.754392 2528 mini_ranger_kms.cc:401] 127.2.120.20:53481/kms/v1/keys
Aug 11 00:29:22 dist-test-slave-fd7l krb5kdc[16037](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786408162, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.120.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 00:29:23 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408162, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.120.20@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
Aug 11 00:29:23 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408162, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.120.20@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
Aug 11 00:29:26 dist-test-slave-fd7l krb5kdc[16037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:29:26.586980 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:42113
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:42113
--ranger_config_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.120.20:53481/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:29:26.980711 16555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:26.981061 16555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:26.981243 16555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:26.989312 16555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:29:26.989444 16555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:26.989528 16555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:26.989620 16555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:29:26.989691 16555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:29:27.003237 16555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.120.20:53481/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:42113
--ranger_config_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:42113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:27.004952 16555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:27.006587 16555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:27.021903 16560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:27.021874 16561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:27.022547 16564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:27.022866 16555 server_base.cc:1034] running on GCE node
I20260811 00:29:27.023833 16555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:27.025677 16555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:27.026975 16555 hybrid_clock.cc:648] HybridClock initialized: now 1786408167026929 us; error 74 us; skew 500 ppm
I20260811 00:29:27.027451 16555 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:27 dist-test-slave-fd7l krb5kdc[16037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408167, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:27.034853 16555 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:29:27.036720 16555 webserver.cc:460] Webserver started at http://127.2.120.62:32941/ using document root <none> and password file <none>
I20260811 00:29:27.037482 16555 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:27.037633 16555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:27.038017 16555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:29:27 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408167, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
I20260811 00:29:27.101922 16555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "a0091eca013744d5aac533e58b14dcdc"
format_stamp: "Formatted at 2026-08-11 00:29:27 on dist-test-slave-fd7l"
server_key: "ebb043ce29fb8c3c021504a40be63855"
server_key_iv: "ce9df60dd0b9e80bb697aec8d2b52743"
server_key_version: "kuduclusterkey@0"
I20260811 00:29:27.103142 16555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "a0091eca013744d5aac533e58b14dcdc"
format_stamp: "Formatted at 2026-08-11 00:29:27 on dist-test-slave-fd7l"
server_key: "ebb043ce29fb8c3c021504a40be63855"
server_key_iv: "ce9df60dd0b9e80bb697aec8d2b52743"
server_key_version: "kuduclusterkey@0"
I20260811 00:29:27.110517 16555 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260811 00:29:27.131131 16572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:27.133152 16555 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 00:29:27.133412 16555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "a0091eca013744d5aac533e58b14dcdc"
format_stamp: "Formatted at 2026-08-11 00:29:27 on dist-test-slave-fd7l"
server_key: "ebb043ce29fb8c3c021504a40be63855"
server_key_iv: "ce9df60dd0b9e80bb697aec8d2b52743"
server_key_version: "kuduclusterkey@0"
I20260811 00:29:27.133713 16555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:27.166375 16555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:27.170943 16555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:27.171325 16555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:27.195257 16555 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:42113
I20260811 00:29:27.195505 16629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:42113 every 8 connection(s)
I20260811 00:29:27.197090 16555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:29:27.201938 16630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:29:27.204442 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 16555
I20260811 00:29:27.204725 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:29:27.205651 2528 external_mini_cluster.cc:1428] Setting key c19a69e403d1a616283f2e8e21cc127f
I20260811 00:29:27.218704 16630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc: Bootstrap starting.
I20260811 00:29:27.225070 16630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc: Neither blocks nor log segments found. Creating new log.
Aug 11 00:29:27 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:29:27.226804 16630 log.cc:824] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc: Log is configured to *not* fsync() on all Append() calls
I20260811 00:29:27.230953 16630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc: No bootstrap required, opened a new log
I20260811 00:29:27.238579 16633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:27.208112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39208 (local address 127.2.120.62:42113)
0811 00:29:27.209448 (+ 1336us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:27.209467 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:27.209518 (+ 51us) server_negotiation.cc:407] Connection header received
0811 00:29:27.211335 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:27.211363 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:27.211834 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:27.212220 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:27.214411 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:27.217273 (+ 2862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:27.219786 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:27.220839 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:27.221060 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:27.227655 (+ 6595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:27.227678 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:27.227711 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:27.227781 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:27.231560 (+ 3779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:27.232718 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:27.232731 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:27.232743 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:27.232968 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:27.233529 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:27.233535 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:27.233538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:27.234223 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:27.234585 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:27.235460 (+ 875us) server_negotiation.cc:299] Negotiation successful
0811 00:29:27.236304 (+ 844us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1038,"thread_start_us":651,"threads_started":1}
I20260811 00:29:27.242267 16630 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0091eca013744d5aac533e58b14dcdc" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42113 } }
I20260811 00:29:27.242846 16630 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:29:27.243079 16630 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0091eca013744d5aac533e58b14dcdc, State: Initialized, Role: FOLLOWER
I20260811 00:29:27.243898 16630 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0091eca013744d5aac533e58b14dcdc" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42113 } }
I20260811 00:29:27.244149 16630 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:29:27.244249 16630 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:29:27.244392 16630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:29:27.246873 16630 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0091eca013744d5aac533e58b14dcdc" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42113 } }
I20260811 00:29:27.247452 16630 leader_election.cc:302] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0091eca013744d5aac533e58b14dcdc; no voters:
I20260811 00:29:27.248359 16630 leader_election.cc:288] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:29:27.248709 16635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:29:27.250356 16635 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [term 1 LEADER]: Becoming Leader. State: Replica: a0091eca013744d5aac533e58b14dcdc, State: Running, Role: LEADER
I20260811 00:29:27.251031 16635 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0091eca013744d5aac533e58b14dcdc" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42113 } }
I20260811 00:29:27.251179 16630 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:29:27.254652 16638 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0091eca013744d5aac533e58b14dcdc. Latest consensus state: current_term: 1 leader_uuid: "a0091eca013744d5aac533e58b14dcdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0091eca013744d5aac533e58b14dcdc" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42113 } } }
I20260811 00:29:27.254725 16637 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0091eca013744d5aac533e58b14dcdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0091eca013744d5aac533e58b14dcdc" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42113 } } }
I20260811 00:29:27.255177 16638 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [sys.catalog]: This master's current role is: LEADER
I20260811 00:29:27.255241 16637 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [sys.catalog]: This master's current role is: LEADER
I20260811 00:29:27.259824 16647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:29:27.266535 16647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:29:27.267395 16630 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 00:29:27.289264 16647 catalog_manager.cc:1347] Generated new cluster ID: e6db4db37fe14d6985efe2ae506669e1
I20260811 00:29:27.289530 16647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:29:27.305290 16647 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:29:27.306872 16647 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:29:27.316563 16647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc: Generated new TSK 0
I20260811 00:29:27.317690 16647 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
I20260811 00:29:28.225744 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.120.20:53481/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:42113
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:29:28 dist-test-slave-fd7l krb5kdc[16037](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786408168, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 00:29:28.710655 16685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:28.711144 16685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:28.711374 16685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:28.724807 16685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:28.725014 16685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:28.725306 16685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:29:28.742916 16685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.120.20:53481/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:42113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:28.745108 16685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:28.747118 16685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:28.770035 16697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:28.776049 16698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:28.786392 16685 server_base.cc:1034] running on GCE node
W20260811 00:29:28.788079 16700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:28.789212 16685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:28.790719 16685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:28.792042 16685 hybrid_clock.cc:648] HybridClock initialized: now 1786408168791956 us; error 73 us; skew 500 ppm
I20260811 00:29:28.792577 16685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:28 dist-test-slave-fd7l krb5kdc[16037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408168, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:28.797670 16685 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:29:28.799921 16685 webserver.cc:460] Webserver started at http://127.2.120.1:37731/ using document root <none> and password file <none>
I20260811 00:29:28.800917 16685 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:28.801120 16685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:28.801488 16685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:29:28 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408168, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
I20260811 00:29:28.825695 16685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "83b2887cfc784979a7c5887d0019f11c"
format_stamp: "Formatted at 2026-08-11 00:29:28 on dist-test-slave-fd7l"
server_key: "05f393392a5e4883530caf14569ffde7"
server_key_iv: "4518a2ee11d5fd84019b7862cb4f0cac"
server_key_version: "kuduclusterkey@0"
I20260811 00:29:28.826918 16685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "83b2887cfc784979a7c5887d0019f11c"
format_stamp: "Formatted at 2026-08-11 00:29:28 on dist-test-slave-fd7l"
server_key: "05f393392a5e4883530caf14569ffde7"
server_key_iv: "4518a2ee11d5fd84019b7862cb4f0cac"
server_key_version: "kuduclusterkey@0"
I20260811 00:29:28.848590 16685 fs_manager.cc:691] Time spent creating directory manager: real 0.021s user 0.008s sys 0.012s
I20260811 00:29:28.893671 16707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:28.896060 16685 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.001s sys 0.010s
I20260811 00:29:28.896451 16685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "83b2887cfc784979a7c5887d0019f11c"
format_stamp: "Formatted at 2026-08-11 00:29:28 on dist-test-slave-fd7l"
server_key: "05f393392a5e4883530caf14569ffde7"
server_key_iv: "4518a2ee11d5fd84019b7862cb4f0cac"
server_key_version: "kuduclusterkey@0"
I20260811 00:29:28.896915 16685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:28.948118 16685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:28.955518 16685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:28.956080 16685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:28.964102 16685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:29:28.973018 16685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:29:28.973272 16685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 00:29:28.973539 16685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:29:28.973694 16685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:29:29 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408168, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
I20260811 00:29:29.570163 16685 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:43483
I20260811 00:29:29.572857 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 16685
I20260811 00:29:29.573211 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:29:29.574259 2528 external_mini_cluster.cc:1428] Setting key 2fd9b913007462a97926853e7cb5d7cd
I20260811 00:29:29.582715 16836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:43483 every 8 connection(s)
I20260811 00:29:29.572252 16685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
WARNING: no policy specified for kudu/127.2.120.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.2@KRBTEST.COM" created.
Aug 11 00:29:29 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408168, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:29:29.694693 16843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:29.608889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:45967 (local address 127.2.120.62:42113)
0811 00:29:29.641007 (+ 32118us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:29.641014 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:29.641043 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:29:29.641143 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:29.641151 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:29.641254 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:29.641408 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:29.656505 (+ 15097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:29.658050 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:29.662662 (+ 4612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:29.663469 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:29.663633 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:29.678074 (+ 14441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:29.678081 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:29.678085 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:29.678130 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:29.681642 (+ 3512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:29.682963 (+ 1321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:29.682970 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:29.682974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:29.683183 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:29.683934 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:29.683940 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:29.683943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:29.684429 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:29.684664 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:29.693904 (+ 9240us) server_negotiation.cc:299] Negotiation successful
0811 00:29:29.694250 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31937,"thread_start_us":35639,"threads_started":1}
I20260811 00:29:29.695973 16844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:29.609329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42113 (local address 127.2.120.1:45967)
0811 00:29:29.635938 (+ 26609us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:29.636030 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:29.639272 (+ 3242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:29.651620 (+ 12348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:29.651644 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:29.652747 (+ 1103us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:29.654068 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:29.654120 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:29.658485 (+ 4365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:29.658496 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:29.662405 (+ 3909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:29.662433 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:29.672914 (+ 10481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:29.672929 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:29.673415 (+ 486us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:29.674463 (+ 1048us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:29.674505 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:29.677602 (+ 3097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:29.681884 (+ 4282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:29.681896 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:29.681919 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:29.682713 (+ 794us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:29.683393 (+ 680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:29.683399 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:29.683404 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:29.683738 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:29.684646 (+ 908us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:29.684657 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:29.693310 (+ 8653us) client_negotiation.cc:769] Sending connection context
0811 00:29:29.693725 (+ 415us) client_negotiation.cc:240] Negotiation successful
0811 00:29:29.694396 (+ 671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26152,"thread_start_us":30424,"threads_started":1}
I20260811 00:29:29.698520 16839 heartbeater.cc:347] Connected to a master server at 127.2.120.62:42113
I20260811 00:29:29.698930 16839 heartbeater.cc:464] Registering TS with master...
I20260811 00:29:29.700038 16839 heartbeater.cc:511] Master 127.2.120.62:42113 requested a full tablet report, sending...
I20260811 00:29:29.704412 16589 ts_manager.cc:194] Registered new tserver with Master: 83b2887cfc784979a7c5887d0019f11c (127.2.120.1:43483)
I20260811 00:29:29.707130 16589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:45967
Aug 11 00:29:29 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408168, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
W20260811 00:29:29.724227 2528 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.2.120.2: real 0.146s user 0.002s sys 0.022s
I20260811 00:29:29.725204 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.120.20:53481/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:42113
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:29:30.184279 16847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:30.184725 16847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:30.184894 16847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:30.193008 16847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:30.193222 16847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:30.193411 16847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:29:30.208853 16847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.120.20:53481/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:42113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:30.210372 16847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:30.211827 16847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:30.230751 16864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:30.230777 16863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:30.230913 16866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:30.231700 16847 server_base.cc:1034] running on GCE node
I20260811 00:29:30.232551 16847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:30.234026 16847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:30.235600 16847 hybrid_clock.cc:648] HybridClock initialized: now 1786408170235485 us; error 102 us; skew 500 ppm
I20260811 00:29:30.236096 16847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:30 dist-test-slave-fd7l krb5kdc[16037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408170, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:30.241086 16847 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:29:30.243008 16847 webserver.cc:460] Webserver started at http://127.2.120.2:44617/ using document root <none> and password file <none>
I20260811 00:29:30.243892 16847 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:30.244061 16847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:30.244454 16847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:29:30 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408170, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
I20260811 00:29:30.266717 16847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "303a3053b4ea4c0bb404b81722d3920d"
format_stamp: "Formatted at 2026-08-11 00:29:30 on dist-test-slave-fd7l"
server_key: "ad3ccb4598458d5e455e21c9bd0a962c"
server_key_iv: "af8b4e5a56ee8662dba41ec570501a26"
server_key_version: "kuduclusterkey@0"
I20260811 00:29:30.268321 16847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "303a3053b4ea4c0bb404b81722d3920d"
format_stamp: "Formatted at 2026-08-11 00:29:30 on dist-test-slave-fd7l"
server_key: "ad3ccb4598458d5e455e21c9bd0a962c"
server_key_iv: "af8b4e5a56ee8662dba41ec570501a26"
server_key_version: "kuduclusterkey@0"
I20260811 00:29:30.279619 16847 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.005s sys 0.004s
I20260811 00:29:30.303252 16874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:30.305580 16847 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260811 00:29:30.305825 16847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "303a3053b4ea4c0bb404b81722d3920d"
format_stamp: "Formatted at 2026-08-11 00:29:30 on dist-test-slave-fd7l"
server_key: "ad3ccb4598458d5e455e21c9bd0a962c"
server_key_iv: "af8b4e5a56ee8662dba41ec570501a26"
server_key_version: "kuduclusterkey@0"
I20260811 00:29:30.306129 16847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:30.328868 16847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:30.336390 16847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:30.337658 16847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:30.339540 16847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:29:30.342046 16847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:29:30.342270 16847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:30.342454 16847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:29:30.342525 16847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:30.383590 16847 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:33493
I20260811 00:29:30.383834 17005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:33493 every 8 connection(s)
I20260811 00:29:30.385216 16847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:29:30.389920 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 16847
I20260811 00:29:30.390129 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:29:30.390678 2528 external_mini_cluster.cc:1428] Setting key 8716e16fb26fa7746f740be39720bc06
Aug 11 00:29:30 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408170, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:29:30.411582 17008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:30.388388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:56499 (local address 127.2.120.62:42113)
0811 00:29:30.389020 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:30.389028 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:30.391343 (+ 2315us) server_negotiation.cc:407] Connection header received
0811 00:29:30.392407 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:30.392412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:30.392572 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:30.392733 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:30.394513 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:30.395356 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:30.397279 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:30.397717 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:30.397875 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:30.401923 (+ 4048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:30.401930 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:30.401934 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:30.401981 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:30.406668 (+ 4687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:30.408081 (+ 1413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:30.408092 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:30.408099 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:30.408324 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:30.409178 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:30.409187 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:30.409193 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:30.409673 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:30.409856 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:30.410761 (+ 905us) server_negotiation.cc:299] Negotiation successful
0811 00:29:30.411082 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":339,"threads_started":1}
I20260811 00:29:30.412441 17009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:30.389010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42113 (local address 127.2.120.2:56499)
0811 00:29:30.389994 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:30.390066 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:30.392016 (+ 1950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:30.393111 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:30.393122 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:30.393622 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:30.394304 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:30.394327 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:30.395549 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:30.395554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:30.397084 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:30.397108 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:30.397996 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:30.398005 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:30.398278 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:30.399142 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:30.399171 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:30.401616 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:30.407021 (+ 5405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:30.407036 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:30.407060 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:30.407820 (+ 760us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:30.408657 (+ 837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:30.408664 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:30.408667 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:30.409002 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:30.409854 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:30.409867 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:30.410286 (+ 419us) client_negotiation.cc:769] Sending connection context
0811 00:29:30.410562 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 00:29:30.411044 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":350,"threads_started":1}
I20260811 00:29:30.414434 17006 heartbeater.cc:347] Connected to a master server at 127.2.120.62:42113
I20260811 00:29:30.414850 17006 heartbeater.cc:464] Registering TS with master...
I20260811 00:29:30.415913 17006 heartbeater.cc:511] Master 127.2.120.62:42113 requested a full tablet report, sending...
I20260811 00:29:30.417955 16589 ts_manager.cc:194] Registered new tserver with Master: 303a3053b4ea4c0bb404b81722d3920d (127.2.120.2:33493)
I20260811 00:29:30.418841 16589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:56499
WARNING: no policy specified for kudu/127.2.120.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
I20260811 00:29:30.461232 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.120.20:53481/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:42113
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:29:30.711802 16839 heartbeater.cc:503] Master 127.2.120.62:42113 was elected leader, sending a full tablet report...
W20260811 00:29:30.845489 17012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:30.845829 17012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:30.845929 17012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:30.854034 17012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:30.854198 17012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:30.854327 17012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:29:30.865283 17012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.120.20:53481/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:42113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:30.866698 17012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:30.868073 17012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:30.884286 17020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:30.884490 17019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:30.884613 17022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:30.886646 17012 server_base.cc:1034] running on GCE node
I20260811 00:29:30.887454 17012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:30.888667 17012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:30.889885 17012 hybrid_clock.cc:648] HybridClock initialized: now 1786408170889822 us; error 66 us; skew 500 ppm
I20260811 00:29:30.890252 17012 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:30 dist-test-slave-fd7l krb5kdc[16037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408170, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:30.893869 17012 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:29:30.895661 17012 webserver.cc:460] Webserver started at http://127.2.120.3:45571/ using document root <none> and password file <none>
I20260811 00:29:30.896406 17012 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:30.896567 17012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:30.896917 17012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:29:30 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408170, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
I20260811 00:29:30.914549 17012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "89a0353c4d4748aea666d5f90978eed4"
format_stamp: "Formatted at 2026-08-11 00:29:30 on dist-test-slave-fd7l"
server_key: "58a10ae55930b23071ada2669ce4b462"
server_key_iv: "b7e6e5604cf103bf25fe0dc9e7ed8457"
server_key_version: "kuduclusterkey@0"
I20260811 00:29:30.915401 17012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "89a0353c4d4748aea666d5f90978eed4"
format_stamp: "Formatted at 2026-08-11 00:29:30 on dist-test-slave-fd7l"
server_key: "58a10ae55930b23071ada2669ce4b462"
server_key_iv: "b7e6e5604cf103bf25fe0dc9e7ed8457"
server_key_version: "kuduclusterkey@0"
I20260811 00:29:30.921923 17012 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 00:29:30.941016 17029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:30.942791 17012 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260811 00:29:30.943022 17012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "89a0353c4d4748aea666d5f90978eed4"
format_stamp: "Formatted at 2026-08-11 00:29:30 on dist-test-slave-fd7l"
server_key: "58a10ae55930b23071ada2669ce4b462"
server_key_iv: "b7e6e5604cf103bf25fe0dc9e7ed8457"
server_key_version: "kuduclusterkey@0"
I20260811 00:29:30.943287 17012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:30.962658 17012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:30.966921 17012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:30.967270 17012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:30.968331 17012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:29:30.969967 17012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:29:30.970084 17012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:30.970249 17012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:29:30.970325 17012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:31.010008 17012 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:40855
I20260811 00:29:31.010111 17159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:40855 every 8 connection(s)
I20260811 00:29:31.011549 17012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:29:31.013381 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 17012
I20260811 00:29:31.013572 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:29:31.014217 2528 external_mini_cluster.cc:1428] Setting key 728b20cf731a981a5b87884cb6ce9e48
Aug 11 00:29:31 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408170, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:29:31.034751 17162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:31.015144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:51303 (local address 127.2.120.62:42113)
0811 00:29:31.015757 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:31.015763 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:31.016962 (+ 1199us) server_negotiation.cc:407] Connection header received
0811 00:29:31.019144 (+ 2182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:31.019150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:31.019239 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:31.019370 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:31.021017 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:31.021772 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:31.023476 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:31.023968 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:31.024127 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:31.027920 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:31.027926 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:31.027930 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:31.027977 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:31.031218 (+ 3241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:31.032166 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:31.032172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:31.032174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:31.032341 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:31.032902 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:31.032907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:31.032911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:31.033279 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:31.033425 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:31.034221 (+ 796us) server_negotiation.cc:299] Negotiation successful
0811 00:29:31.034446 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":303,"threads_started":1}
I20260811 00:29:31.035792 17163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:31.015833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42113 (local address 127.2.120.3:51303)
0811 00:29:31.016798 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:31.016841 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:31.018846 (+ 2005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:31.019659 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:31.019668 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:31.020121 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:31.020832 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:31.020854 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:31.021948 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:31.021952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:31.023301 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:31.023316 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:31.024226 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:31.024235 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:31.024571 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:31.025310 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:31.025336 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:31.027639 (+ 2303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:31.031399 (+ 3760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:31.031409 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:31.031427 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:31.031987 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:31.032463 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:31.032468 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:31.032471 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:31.032776 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:31.033433 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:31.033440 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:31.033817 (+ 377us) client_negotiation.cc:769] Sending connection context
0811 00:29:31.034099 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 00:29:31.034554 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":348,"threads_started":1}
I20260811 00:29:31.037523 17160 heartbeater.cc:347] Connected to a master server at 127.2.120.62:42113
I20260811 00:29:31.037919 17160 heartbeater.cc:464] Registering TS with master...
I20260811 00:29:31.038761 17160 heartbeater.cc:511] Master 127.2.120.62:42113 requested a full tablet report, sending...
I20260811 00:29:31.040478 16589 ts_manager.cc:194] Registered new tserver with Master: 89a0353c4d4748aea666d5f90978eed4 (127.2.120.3:40855)
I20260811 00:29:31.041647 16589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:51303
I20260811 00:29:31.043668 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:29:31.045562 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 16685
Aug 11 00:29:31 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:29:31.065706 17165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:31.046610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50200 (local address 127.2.120.1:43483)
0811 00:29:31.047414 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:31.047431 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:31.047463 (+ 32us) server_negotiation.cc:407] Connection header received
0811 00:29:31.047529 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:31.047535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:31.047813 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:31.047971 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:31.049807 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:31.050977 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:31.052823 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:31.053515 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:31.053655 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:31.058461 (+ 4806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:31.058480 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:31.058498 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:31.058546 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:31.061964 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:31.062911 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:31.062920 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:31.062922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:31.063090 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:31.063918 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:31.063922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:31.063924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:31.064342 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:31.064717 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:31.065104 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 00:29:31.065376 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":471,"threads_started":1}
I20260811 00:29:31.067250 16832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:31.160784 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 16685
I20260811 00:29:31.181591 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 16847
Aug 11 00:29:31 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.2@KRBTEST.COM
I20260811 00:29:31.207065 17168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:31.187337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55596 (local address 127.2.120.2:33493)
0811 00:29:31.187955 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:31.187963 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:31.187993 (+ 30us) server_negotiation.cc:407] Connection header received
0811 00:29:31.188064 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:31.188071 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:31.188284 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:31.188441 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:31.189975 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:31.190999 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:31.192720 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:31.193308 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:31.193438 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:31.198256 (+ 4818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:31.198276 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:31.198299 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:31.198350 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:31.203400 (+ 5050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:31.204313 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:31.204325 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:31.204330 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:31.204588 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:31.205026 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:31.205032 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:31.205036 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:31.205541 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:31.205876 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:31.206275 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 00:29:31.206595 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":314,"threads_started":1}
I20260811 00:29:31.208683 16999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:31.295586 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 16847
I20260811 00:29:31.316337 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 17012
Aug 11 00:29:31 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.3@KRBTEST.COM
I20260811 00:29:31.337488 17171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:31.317484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53244 (local address 127.2.120.3:40855)
0811 00:29:31.318201 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:31.318209 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:31.318242 (+ 33us) server_negotiation.cc:407] Connection header received
0811 00:29:31.318307 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:31.318313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:31.318556 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:31.318696 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:31.320112 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:31.321667 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:31.323265 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:31.324173 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:31.324647 (+ 474us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:31.330112 (+ 5465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:31.330132 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:31.330153 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:31.330204 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:31.334131 (+ 3927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:31.335087 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:31.335096 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:31.335099 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:31.335260 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:31.335830 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:31.335835 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:31.335837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:31.336262 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:31.336610 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:31.336950 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 00:29:31.337187 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":395,"threads_started":1}
I20260811 00:29:31.338874 17155 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:31.426138 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 17012
I20260811 00:29:31.447901 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 16555
I20260811 00:29:31.449491 16625 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:31.567405 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 16555
I20260811 00:29:31.603008 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:42113
--webserver_interface=127.2.120.62
--webserver_port=32941
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:42113
--ranger_config_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.120.20:53481/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.120.20:53481/kms with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:29:31.990736 17174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:31.991250 17174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:31.991436 17174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:32.002872 17174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:29:32.003060 17174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:32.003144 17174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:32.003230 17174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:29:32.003304 17174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:29:32.014514 17174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.120.20:53481/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:42113
--ranger_config_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:42113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=32941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:32.015892 17174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:32.017350 17174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:32.030953 17181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:32.030977 17184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:32.031006 17179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:32.031987 17174 server_base.cc:1034] running on GCE node
I20260811 00:29:32.033013 17174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:32.034737 17174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:32.036044 17174 hybrid_clock.cc:648] HybridClock initialized: now 1786408172035980 us; error 53 us; skew 500 ppm
I20260811 00:29:32.036626 17174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:32 dist-test-slave-fd7l krb5kdc[16037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408172, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:32.046463 17174 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:29:32.048593 17174 webserver.cc:460] Webserver started at http://127.2.120.62:32941/ using document root <none> and password file <none>
I20260811 00:29:32.049695 17174 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:32.049877 17174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 00:29:32 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408172, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
I20260811 00:29:32.323580 17174 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:29:32.328685 17192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:32.330852 17174 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260811 00:29:32.331120 17174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "a0091eca013744d5aac533e58b14dcdc"
format_stamp: "Formatted at 2026-08-11 00:29:27 on dist-test-slave-fd7l"
server_key: "ebb043ce29fb8c3c021504a40be63855"
server_key_iv: "ce9df60dd0b9e80bb697aec8d2b52743"
server_key_version: "kuduclusterkey@0"
I20260811 00:29:32.332719 17174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:32.349946 17174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:32.354823 17174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:32.355104 17174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:32.377002 17174 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:42113
I20260811 00:29:32.377041 17251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:42113 every 8 connection(s)
I20260811 00:29:32.378679 17174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:29:32.382074 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 17174
I20260811 00:29:32.382985 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.120.20:53481/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:43483
--local_ip_for_outbound_sockets=127.2.120.1
--tserver_master_addrs=127.2.120.62:42113
--webserver_port=37731
--webserver_interface=127.2.120.1
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.120.20:53481/kms with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:29:32.386471 17252 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:29:32.390123 17252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc: Bootstrap starting.
I20260811 00:29:32.411521 17252 log.cc:824] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc: Log is configured to *not* fsync() on all Append() calls
I20260811 00:29:32.421708 17252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:29:32.422359 17252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc: Bootstrap complete.
I20260811 00:29:32.432066 17252 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0091eca013744d5aac533e58b14dcdc" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42113 } }
I20260811 00:29:32.433147 17252 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0091eca013744d5aac533e58b14dcdc, State: Initialized, Role: FOLLOWER
I20260811 00:29:32.433840 17252 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0091eca013744d5aac533e58b14dcdc" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42113 } }
I20260811 00:29:32.434141 17252 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:29:32.434293 17252 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:29:32.434504 17252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [term 1 FOLLOWER]: Advancing to term 2
I20260811 00:29:32.438827 17252 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0091eca013744d5aac533e58b14dcdc" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42113 } }
I20260811 00:29:32.439442 17252 leader_election.cc:302] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0091eca013744d5aac533e58b14dcdc; no voters:
I20260811 00:29:32.440173 17252 leader_election.cc:288] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 00:29:32.440441 17256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [term 2 FOLLOWER]: Leader election won for term 2
I20260811 00:29:32.441866 17256 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [term 2 LEADER]: Becoming Leader. State: Replica: a0091eca013744d5aac533e58b14dcdc, State: Running, Role: LEADER
I20260811 00:29:32.442828 17256 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0091eca013744d5aac533e58b14dcdc" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42113 } }
I20260811 00:29:32.443462 17252 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:29:32.446234 17258 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0091eca013744d5aac533e58b14dcdc. Latest consensus state: current_term: 2 leader_uuid: "a0091eca013744d5aac533e58b14dcdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0091eca013744d5aac533e58b14dcdc" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42113 } } }
I20260811 00:29:32.446763 17258 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [sys.catalog]: This master's current role is: LEADER
I20260811 00:29:32.446233 17257 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a0091eca013744d5aac533e58b14dcdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0091eca013744d5aac533e58b14dcdc" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42113 } } }
I20260811 00:29:32.447768 17257 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc [sys.catalog]: This master's current role is: LEADER
I20260811 00:29:32.448904 17267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:29:32.468108 17267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:29:32.470221 17267 catalog_manager.cc:1259] Loaded cluster ID: e6db4db37fe14d6985efe2ae506669e1
I20260811 00:29:32.470361 17267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:29:32.476013 17267 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:29:32.478286 17267 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc: Loaded TSK: 0
I20260811 00:29:32.480434 17267 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:29:32.493628 17252 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260811 00:29:32.926849 17254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:32.927311 17254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:32.927517 17254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:32.940011 17254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:32.940318 17254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:32.940611 17254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:29:32.958627 17254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.120.20:53481/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:43483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=37731
--tserver_master_addrs=127.2.120.62:42113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:32.960646 17254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:32.962815 17254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:33.002933 17299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:33.003578 17298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:33.006259 17301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:33.006793 17254 server_base.cc:1034] running on GCE node
I20260811 00:29:33.007735 17254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:33.009367 17254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:33.010716 17254 hybrid_clock.cc:648] HybridClock initialized: now 1786408173010630 us; error 60 us; skew 500 ppm
I20260811 00:29:33.011186 17254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:33 dist-test-slave-fd7l krb5kdc[16037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408173, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:33.016163 17254 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:29:33.018859 17254 webserver.cc:460] Webserver started at http://127.2.120.1:37731/ using document root <none> and password file <none>
I20260811 00:29:33.020071 17254 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:33.020346 17254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 00:29:33 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408173, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
I20260811 00:29:33.057055 17254 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 00:29:33.062598 17312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:33.064458 17254 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:29:33.064746 17254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "83b2887cfc784979a7c5887d0019f11c"
format_stamp: "Formatted at 2026-08-11 00:29:28 on dist-test-slave-fd7l"
server_key: "05f393392a5e4883530caf14569ffde7"
server_key_iv: "4518a2ee11d5fd84019b7862cb4f0cac"
server_key_version: "kuduclusterkey@0"
I20260811 00:29:33.066077 17254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:33.097913 17254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:33.103322 17254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:33.103725 17254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:33.105149 17254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:29:33.107164 17254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:29:33.107318 17254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:33.107549 17254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:29:33.107734 17254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:33.166149 17254 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:43483
I20260811 00:29:33.166203 17439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:43483 every 8 connection(s)
I20260811 00:29:33.168037 17254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:29:33.171448 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 17254
I20260811 00:29:33.172336 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.120.20:53481/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:33493
--local_ip_for_outbound_sockets=127.2.120.2
--tserver_master_addrs=127.2.120.62:42113
--webserver_port=44617
--webserver_interface=127.2.120.2
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.120.20:53481/kms with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:29:33 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408173, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:29:33.221468 17442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:33.172252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:59329 (local address 127.2.120.62:42113)
0811 00:29:33.187350 (+ 15098us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:33.187372 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:33.187439 (+ 67us) server_negotiation.cc:407] Connection header received
0811 00:29:33.190565 (+ 3126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:33.190591 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:33.191149 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:33.191583 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:33.194593 (+ 3010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:33.196777 (+ 2184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:33.199703 (+ 2926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:33.200945 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:33.201218 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:33.206202 (+ 4984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:33.206212 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:33.206231 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:33.206295 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:33.210931 (+ 4636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:33.212447 (+ 1516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:33.212459 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:33.212469 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:33.212780 (+ 311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:33.213535 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:33.213542 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:33.213546 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:33.214441 (+ 895us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:33.214695 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:33.219518 (+ 4823us) server_negotiation.cc:299] Negotiation successful
0811 00:29:33.220076 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14763,"thread_start_us":15074,"threads_started":1}
I20260811 00:29:33.221488 17444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:33.172391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42113 (local address 127.2.120.1:59329)
0811 00:29:33.185765 (+ 13374us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:33.185850 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:33.189719 (+ 3869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:33.192571 (+ 2852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:33.192587 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:33.193321 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:33.194220 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:33.194255 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:33.197075 (+ 2820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:33.197082 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:33.199390 (+ 2308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:33.199432 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:33.201292 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:33.201303 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:33.201645 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:33.202575 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:33.202630 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:33.205786 (+ 3156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:33.211268 (+ 5482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:33.211284 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:33.211308 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:33.212169 (+ 861us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:33.212973 (+ 804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:33.212981 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:33.212987 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:33.213341 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:33.217964 (+ 4623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:33.217981 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:33.218648 (+ 667us) client_negotiation.cc:769] Sending connection context
0811 00:29:33.219099 (+ 451us) client_negotiation.cc:240] Negotiation successful
0811 00:29:33.219849 (+ 750us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12914,"thread_start_us":15500,"threads_started":1}
I20260811 00:29:33.226538 17440 heartbeater.cc:347] Connected to a master server at 127.2.120.62:42113
I20260811 00:29:33.226976 17440 heartbeater.cc:464] Registering TS with master...
I20260811 00:29:33.228070 17440 heartbeater.cc:511] Master 127.2.120.62:42113 requested a full tablet report, sending...
W20260811 00:29:33.232476 17440 heartbeater.cc:644] Failed to heartbeat to 127.2.120.62:42113 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 00:29:33.232986 17440 heartbeater.cc:464] Registering TS with master...
I20260811 00:29:33.233458 17440 heartbeater.cc:511] Master 127.2.120.62:42113 requested a full tablet report, sending...
I20260811 00:29:33.235178 17440 heartbeater.cc:464] Registering TS with master...
I20260811 00:29:33.235599 17440 heartbeater.cc:511] Master 127.2.120.62:42113 requested a full tablet report, sending...
W20260811 00:29:33.237170 17440 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:29:33.647007 17445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:33.647526 17445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:33.647756 17445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:33.657902 17445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:33.658102 17445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:33.658309 17445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:29:33.671787 17445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.120.20:53481/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:33493
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=44617
--tserver_master_addrs=127.2.120.62:42113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:33.673427 17445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:33.674909 17445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:33.694144 17463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:33.694123 17461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:33.698837 17460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:33.706137 17445 server_base.cc:1034] running on GCE node
I20260811 00:29:33.707098 17445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:33.708564 17445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:33.709841 17445 hybrid_clock.cc:648] HybridClock initialized: now 1786408173709781 us; error 48 us; skew 500 ppm
I20260811 00:29:33.710246 17445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:33 dist-test-slave-fd7l krb5kdc[16037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408173, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:33.714927 17445 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:29:33.720541 17445 webserver.cc:460] Webserver started at http://127.2.120.2:44617/ using document root <none> and password file <none>
I20260811 00:29:33.721733 17445 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:33.721936 17445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 00:29:33 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408173, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
I20260811 00:29:33.755609 17445 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.001s sys 0.004s
I20260811 00:29:33.760054 17473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:33.762272 17445 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:29:33.762494 17445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "303a3053b4ea4c0bb404b81722d3920d"
format_stamp: "Formatted at 2026-08-11 00:29:30 on dist-test-slave-fd7l"
server_key: "ad3ccb4598458d5e455e21c9bd0a962c"
server_key_iv: "af8b4e5a56ee8662dba41ec570501a26"
server_key_version: "kuduclusterkey@0"
I20260811 00:29:33.764024 17445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:33.829777 17445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:33.835995 17445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:33.836416 17445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:33.844476 17445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:29:33.853259 17445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:29:33.853494 17445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 00:29:33.853693 17445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:29:33.853848 17445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:29:33 dist-test-slave-fd7l krb5kdc[16037](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786408173, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:33.925493 17445 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:33493
I20260811 00:29:33.925627 17606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:33493 every 8 connection(s)
I20260811 00:29:33.927129 17445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:29:33.932754 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 17445
I20260811 00:29:33.933646 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.120.20:53481/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:40855
--local_ip_for_outbound_sockets=127.2.120.3
--tserver_master_addrs=127.2.120.62:42113
--webserver_port=45571
--webserver_interface=127.2.120.3
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.120.20:53481/kms with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:29:33 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408173, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:29:33.960276 17609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:33.930410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:58329 (local address 127.2.120.62:42113)
0811 00:29:33.930970 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:33.930977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:33.932035 (+ 1058us) server_negotiation.cc:407] Connection header received
0811 00:29:33.934333 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:33.934339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:33.934465 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:33.934578 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:33.937093 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:33.938423 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:33.941781 (+ 3358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:33.942570 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:33.942692 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:33.947797 (+ 5105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:33.947808 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:33.947814 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:33.947865 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:33.953929 (+ 6064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:33.955479 (+ 1550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:33.955496 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:33.955499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:33.955718 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:33.956442 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:33.956448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:33.956452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:33.956956 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:33.957131 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:33.959152 (+ 2021us) server_negotiation.cc:299] Negotiation successful
0811 00:29:33.959545 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":425,"threads_started":1}
I20260811 00:29:33.964159 17611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:33.930933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42113 (local address 127.2.120.2:58329)
0811 00:29:33.931822 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:33.931872 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:33.933964 (+ 2092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:33.935097 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:33.935125 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:33.935844 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:33.936810 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:33.936848 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:33.938773 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:33.938779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:33.941506 (+ 2727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:33.941544 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:33.943183 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:33.943192 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:33.943541 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:33.944537 (+ 996us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:33.944580 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:33.947405 (+ 2825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:33.954249 (+ 6844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:33.954267 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:33.954299 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:33.955212 (+ 913us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:33.955920 (+ 708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:33.955925 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:33.955929 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:33.956268 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:33.957250 (+ 982us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:33.957261 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:33.958245 (+ 984us) client_negotiation.cc:769] Sending connection context
0811 00:29:33.961072 (+ 2827us) client_negotiation.cc:240] Negotiation successful
0811 00:29:33.961931 (+ 859us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":365,"threads_started":1}
I20260811 00:29:33.966903 17607 heartbeater.cc:347] Connected to a master server at 127.2.120.62:42113
I20260811 00:29:33.967325 17607 heartbeater.cc:464] Registering TS with master...
I20260811 00:29:33.968472 17607 heartbeater.cc:511] Master 127.2.120.62:42113 requested a full tablet report, sending...
I20260811 00:29:33.971567 17211 ts_manager.cc:194] Registered new tserver with Master: 303a3053b4ea4c0bb404b81722d3920d (127.2.120.2:33493)
I20260811 00:29:33.974184 17211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:58329
I20260811 00:29:34.237329 17440 heartbeater.cc:464] Registering TS with master...
I20260811 00:29:34.237800 17440 heartbeater.cc:511] Master 127.2.120.62:42113 requested a full tablet report, sending...
I20260811 00:29:34.239169 17211 ts_manager.cc:194] Registered new tserver with Master: 83b2887cfc784979a7c5887d0019f11c (127.2.120.1:43483)
I20260811 00:29:34.240221 17211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:59329
W20260811 00:29:34.436877 17613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:34.437395 17613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:34.437587 17613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Aug 11 00:29:34 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408173, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
W20260811 00:29:34.447497 17613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:34.447700 17613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:34.447918 17613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:29:34.464066 17613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.120.20:53481/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:40855
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=45571
--tserver_master_addrs=127.2.120.62:42113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:34.466127 17613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:34.468040 17613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:34.527426 17621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:34.527781 17620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:34.540747 17623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:34.540978 17613 server_base.cc:1034] running on GCE node
I20260811 00:29:34.541759 17613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:34.542982 17613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:34.544271 17613 hybrid_clock.cc:648] HybridClock initialized: now 1786408174544209 us; error 57 us; skew 500 ppm
I20260811 00:29:34.544723 17613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:34 dist-test-slave-fd7l krb5kdc[16037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408174, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:34.548195 17613 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:29:34.549940 17613 webserver.cc:460] Webserver started at http://127.2.120.3:45571/ using document root <none> and password file <none>
I20260811 00:29:34.550710 17613 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:34.550873 17613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 00:29:34 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408174, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
I20260811 00:29:34.572625 17613 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:29:34.577176 17633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:34.579218 17613 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:29:34.579442 17613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "89a0353c4d4748aea666d5f90978eed4"
format_stamp: "Formatted at 2026-08-11 00:29:30 on dist-test-slave-fd7l"
server_key: "58a10ae55930b23071ada2669ce4b462"
server_key_iv: "b7e6e5604cf103bf25fe0dc9e7ed8457"
server_key_version: "kuduclusterkey@0"
I20260811 00:29:34.580996 17613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:34.598832 17613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:34.604413 17613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:34.604796 17613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:34.606143 17613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:29:34.608193 17613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:29:34.608315 17613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:34.608443 17613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:29:34.608553 17613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:34.662309 17613 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:40855
I20260811 00:29:34.662376 17761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:40855 every 8 connection(s)
I20260811 00:29:34.664613 17613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:29:34.668941 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 17613
Aug 11 00:29:34 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408174, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:29:34.707571 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:34.682522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43560 (local address 127.2.120.62:42113)
0811 00:29:34.684603 (+ 2081us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:34.684798 (+ 195us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:34.684823 (+ 25us) server_negotiation.cc:407] Connection header received
0811 00:29:34.684881 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:34.684886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:34.684974 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:34.687473 (+ 2499us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:34.688224 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.689545 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.691959 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.693487 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.697643 (+ 4156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:34.697994 (+ 351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:34.698003 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:34.698008 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:34.698086 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:34.702732 (+ 4646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:34.703818 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:34.703823 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:34.703827 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:34.704042 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:34.704625 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:34.704630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:34.704633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:34.705095 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:34.706780 (+ 1685us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:34.706909 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 00:29:34.707162 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1908,"thread_start_us":1544,"threads_started":1}
I20260811 00:29:34.711107 17766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:34.668967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42113 (local address 127.2.120.3:46879)
0811 00:29:34.678229 (+ 9262us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:34.678322 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:34.682828 (+ 4506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:34.685369 (+ 2541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:34.685384 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:34.686156 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:34.687424 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:34.687476 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.689618 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.689624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:34.691850 (+ 2226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:34.691873 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.693199 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.693207 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:34.693522 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:34.694449 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:34.694483 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:34.697385 (+ 2902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:34.705757 (+ 8372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:34.705769 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:34.705793 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:34.706504 (+ 711us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:34.706993 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:34.706998 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:34.707001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:34.707284 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:34.708073 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:34.708082 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:34.708636 (+ 554us) client_negotiation.cc:769] Sending connection context
0811 00:29:34.708980 (+ 344us) client_negotiation.cc:240] Negotiation successful
0811 00:29:34.709585 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8739,"thread_start_us":8377,"threads_started":1}
I20260811 00:29:34.712652 17765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:34.667912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:46879 (local address 127.2.120.62:42113)
0811 00:29:34.682550 (+ 14638us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:34.682558 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:34.682585 (+ 27us) server_negotiation.cc:407] Connection header received
0811 00:29:34.684276 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:34.684282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:34.684391 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:34.684583 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:34.687827 (+ 3244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.689193 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.692089 (+ 2896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.692884 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.693041 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:34.697768 (+ 4727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:34.697777 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:34.697782 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:34.697872 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:34.705525 (+ 7653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:34.706697 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:34.706702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:34.706709 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:34.706849 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:34.707430 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:34.707435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:34.707438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:34.707809 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:34.707935 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:34.711903 (+ 3968us) server_negotiation.cc:299] Negotiation successful
0811 00:29:34.712182 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":14498,"spinlock_wait_cycles":1920,"thread_start_us":14347,"threads_started":1}
I20260811 00:29:34.714365 17762 heartbeater.cc:347] Connected to a master server at 127.2.120.62:42113
I20260811 00:29:34.714778 17762 heartbeater.cc:464] Registering TS with master...
I20260811 00:29:34.715847 17762 heartbeater.cc:511] Master 127.2.120.62:42113 requested a full tablet report, sending...
I20260811 00:29:34.717903 17211 ts_manager.cc:194] Registered new tserver with Master: 89a0353c4d4748aea666d5f90978eed4 (127.2.120.3:40855)
I20260811 00:29:34.719054 17211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:46879
I20260811 00:29:34.724614 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:29:34.758497 17765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:34.728459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43576 (local address 127.2.120.62:42113)
0811 00:29:34.728688 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:34.728694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:34.737304 (+ 8610us) server_negotiation.cc:407] Connection header received
0811 00:29:34.737518 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:34.737525 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:34.737631 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:34.737754 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:34.740184 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.741380 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.745817 (+ 4437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.746545 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.746710 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:34.750857 (+ 4147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:34.750865 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:34.750870 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:34.750924 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:34.754653 (+ 3729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:34.755696 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:34.755703 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:34.755707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:34.755900 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:34.756446 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:34.756451 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:34.756455 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:34.756847 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:34.757106 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:34.757774 (+ 668us) server_negotiation.cc:299] Negotiation successful
0811 00:29:34.758031 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 00:29:34.766436 17211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:29:34.779656 17211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 11 00:29:34 dist-test-slave-fd7l krb5kdc[16037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408173, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for HTTP/127.2.120.20@KRBTEST.COM
I20260811 00:29:34.856621 17785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:34.834827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60864 (local address 127.2.120.1:43483)
0811 00:29:34.835505 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:34.835516 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:34.845548 (+ 10032us) server_negotiation.cc:407] Connection header received
0811 00:29:34.845901 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:34.845917 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:34.846406 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:34.846631 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:29:34.847720 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.849594 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.852913 (+ 3319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.854609 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.854736 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:34.855040 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:34.856000 (+ 960us) server_negotiation.cc:299] Negotiation successful
0811 00:29:34.856226 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":289,"threads_started":1}
I20260811 00:29:34.857920 17787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:34.834235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43483 (local address 127.0.0.1:60864)
0811 00:29:34.845272 (+ 11037us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:34.845320 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:34.845553 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:34.846953 (+ 1400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:34.846960 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:34.846994 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:29:34.847462 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:34.847477 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.849812 (+ 2335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.849818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:34.852640 (+ 2822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:34.852667 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.855307 (+ 2640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.855315 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:34.855658 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:34.855726 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 00:29:34.857203 (+ 1477us) client_negotiation.cc:240] Negotiation successful
0811 00:29:34.857531 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10722,"threads_started":1}
I20260811 00:29:34.873382 17366 tablet_service.cc:1467] Processing CreateTablet for tablet ea35a40b80e5439e90e7393961f214cf (DEFAULT_TABLE table=test-table [id=e4e282968c8f4419bacec192824d00a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:29:34.875234 17366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea35a40b80e5439e90e7393961f214cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:29:34.876874 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:34.846773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:40855 (local address 127.0.0.1:37774)
0811 00:29:34.865148 (+ 18375us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:34.865176 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:34.865345 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:34.867367 (+ 2022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:34.867375 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:34.867416 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:29:34.867973 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:34.867985 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.871397 (+ 3412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.871403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:34.873589 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:34.873609 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.875905 (+ 2296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.875913 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:34.876123 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:34.876163 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 00:29:34.876273 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 00:29:34.876444 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18179,"thread_start_us":17984,"threads_started":1}
I20260811 00:29:34.877753 17790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:34.847445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37774 (local address 127.2.120.3:40855)
0811 00:29:34.866123 (+ 18678us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:34.866137 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:34.866185 (+ 48us) server_negotiation.cc:407] Connection header received
0811 00:29:34.866329 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:34.866343 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:34.866720 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:34.866973 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:29:34.868611 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.871073 (+ 2462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.873821 (+ 2748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.875322 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.875468 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:34.875718 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:34.877196 (+ 1478us) server_negotiation.cc:299] Negotiation successful
0811 00:29:34.877392 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18403,"thread_start_us":18211,"threads_started":1}
I20260811 00:29:34.882550 17687 tablet_service.cc:1467] Processing CreateTablet for tablet ea35a40b80e5439e90e7393961f214cf (DEFAULT_TABLE table=test-table [id=e4e282968c8f4419bacec192824d00a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:29:34.884469 17687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea35a40b80e5439e90e7393961f214cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:29:34.886993 17787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:34.837991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:33493 (local address 127.0.0.1:45052)
0811 00:29:34.858423 (+ 20432us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:34.858444 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:34.858567 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:34.879308 (+ 20741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:34.879314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:34.879355 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:29:34.879574 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:34.879594 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.882051 (+ 2457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.882058 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:34.883947 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:34.883972 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.886084 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.886092 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:34.886357 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:34.886410 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 00:29:34.886501 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 00:29:34.886675 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20302,"thread_start_us":6776,"threads_started":1}
I20260811 00:29:34.888670 17788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:34.838171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45052 (local address 127.2.120.2:33493)
0811 00:29:34.873849 (+ 35678us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:34.873858 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:34.873889 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:29:34.873997 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:34.874004 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:34.874275 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:34.879804 (+ 5529us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:29:34.880088 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.881861 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.884235 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:34.885851 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:34.887424 (+ 1573us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:34.887649 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:34.887726 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 00:29:34.887906 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35473,"thread_start_us":36240,"threads_started":1}
I20260811 00:29:34.893028 17529 tablet_service.cc:1467] Processing CreateTablet for tablet ea35a40b80e5439e90e7393961f214cf (DEFAULT_TABLE table=test-table [id=e4e282968c8f4419bacec192824d00a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:29:34.894703 17529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea35a40b80e5439e90e7393961f214cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:29:34.911085 17792 tablet_bootstrap.cc:492] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c: Bootstrap starting.
I20260811 00:29:34.915293 17792 tablet_bootstrap.cc:654] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c: Neither blocks nor log segments found. Creating new log.
I20260811 00:29:34.916843 17792 log.cc:824] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c: Log is configured to *not* fsync() on all Append() calls
I20260811 00:29:34.919595 17793 tablet_bootstrap.cc:492] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4: Bootstrap starting.
I20260811 00:29:34.924856 17793 tablet_bootstrap.cc:654] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4: Neither blocks nor log segments found. Creating new log.
I20260811 00:29:34.925868 17795 tablet_bootstrap.cc:492] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d: Bootstrap starting.
I20260811 00:29:34.927465 17792 tablet_bootstrap.cc:492] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c: No bootstrap required, opened a new log
I20260811 00:29:34.927799 17792 ts_tablet_manager.cc:1397] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.004s
I20260811 00:29:34.939477 17792 raft_consensus.cc:348] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b2887cfc784979a7c5887d0019f11c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43483 } } peers { permanent_uuid: "303a3053b4ea4c0bb404b81722d3920d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33493 } } peers { permanent_uuid: "89a0353c4d4748aea666d5f90978eed4" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 40855 } }
I20260811 00:29:34.940232 17792 raft_consensus.cc:374] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:29:34.940382 17792 raft_consensus.cc:729] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83b2887cfc784979a7c5887d0019f11c, State: Initialized, Role: FOLLOWER
I20260811 00:29:34.941264 17792 consensus_queue.cc:260] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b2887cfc784979a7c5887d0019f11c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43483 } } peers { permanent_uuid: "303a3053b4ea4c0bb404b81722d3920d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33493 } } peers { permanent_uuid: "89a0353c4d4748aea666d5f90978eed4" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 40855 } }
I20260811 00:29:34.947346 17795 tablet_bootstrap.cc:654] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d: Neither blocks nor log segments found. Creating new log.
I20260811 00:29:34.947729 17793 log.cc:824] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4: Log is configured to *not* fsync() on all Append() calls
I20260811 00:29:34.951646 17795 log.cc:824] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d: Log is configured to *not* fsync() on all Append() calls
I20260811 00:29:34.955004 17440 heartbeater.cc:503] Master 127.2.120.62:42113 was elected leader, sending a full tablet report...
I20260811 00:29:34.958906 17792 ts_tablet_manager.cc:1428] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c: Time spent starting tablet: real 0.031s user 0.012s sys 0.016s
I20260811 00:29:34.964754 17793 tablet_bootstrap.cc:492] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4: No bootstrap required, opened a new log
I20260811 00:29:34.965117 17793 ts_tablet_manager.cc:1397] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4: Time spent bootstrapping tablet: real 0.046s user 0.009s sys 0.003s
I20260811 00:29:34.976430 17793 raft_consensus.cc:348] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b2887cfc784979a7c5887d0019f11c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43483 } } peers { permanent_uuid: "303a3053b4ea4c0bb404b81722d3920d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33493 } } peers { permanent_uuid: "89a0353c4d4748aea666d5f90978eed4" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 40855 } }
I20260811 00:29:34.977166 17793 raft_consensus.cc:374] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:29:34.977344 17793 raft_consensus.cc:729] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89a0353c4d4748aea666d5f90978eed4, State: Initialized, Role: FOLLOWER
I20260811 00:29:34.978348 17793 consensus_queue.cc:260] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b2887cfc784979a7c5887d0019f11c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43483 } } peers { permanent_uuid: "303a3053b4ea4c0bb404b81722d3920d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33493 } } peers { permanent_uuid: "89a0353c4d4748aea666d5f90978eed4" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 40855 } }
I20260811 00:29:34.980381 17795 tablet_bootstrap.cc:492] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d: No bootstrap required, opened a new log
I20260811 00:29:34.980831 17607 heartbeater.cc:503] Master 127.2.120.62:42113 was elected leader, sending a full tablet report...
I20260811 00:29:34.980876 17795 ts_tablet_manager.cc:1397] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d: Time spent bootstrapping tablet: real 0.055s user 0.007s sys 0.019s
I20260811 00:29:34.992322 17762 heartbeater.cc:503] Master 127.2.120.62:42113 was elected leader, sending a full tablet report...
I20260811 00:29:34.997812 17793 ts_tablet_manager.cc:1428] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4: Time spent starting tablet: real 0.033s user 0.022s sys 0.005s
I20260811 00:29:34.997967 17795 raft_consensus.cc:348] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b2887cfc784979a7c5887d0019f11c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43483 } } peers { permanent_uuid: "303a3053b4ea4c0bb404b81722d3920d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33493 } } peers { permanent_uuid: "89a0353c4d4748aea666d5f90978eed4" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 40855 } }
I20260811 00:29:34.998585 17795 raft_consensus.cc:374] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:29:34.998739 17795 raft_consensus.cc:729] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 303a3053b4ea4c0bb404b81722d3920d, State: Initialized, Role: FOLLOWER
I20260811 00:29:34.999657 17795 consensus_queue.cc:260] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b2887cfc784979a7c5887d0019f11c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43483 } } peers { permanent_uuid: "303a3053b4ea4c0bb404b81722d3920d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33493 } } peers { permanent_uuid: "89a0353c4d4748aea666d5f90978eed4" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 40855 } }
I20260811 00:29:35.003283 17795 ts_tablet_manager.cc:1428] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d: Time spent starting tablet: real 0.022s user 0.015s sys 0.000s
I20260811 00:29:35.047701 17796 raft_consensus.cc:482] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:29:35.048105 17796 raft_consensus.cc:504] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b2887cfc784979a7c5887d0019f11c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43483 } } peers { permanent_uuid: "303a3053b4ea4c0bb404b81722d3920d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33493 } } peers { permanent_uuid: "89a0353c4d4748aea666d5f90978eed4" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 40855 } }
I20260811 00:29:35.051964 17796 leader_election.cc:288] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 303a3053b4ea4c0bb404b81722d3920d (127.2.120.2:33493), 89a0353c4d4748aea666d5f90978eed4 (127.2.120.3:40855)
I20260811 00:29:35.072913 17790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:35.054076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:58305 (local address 127.2.120.3:40855)
0811 00:29:35.054281 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:35.054287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:35.065233 (+ 10946us) server_negotiation.cc:407] Connection header received
0811 00:29:35.065459 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:35.065465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:35.065548 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:35.065672 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:29:35.066391 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:35.067512 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:35.070263 (+ 2751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:35.071532 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:35.071641 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:35.071757 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:35.072413 (+ 656us) server_negotiation.cc:299] Negotiation successful
0811 00:29:35.072605 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 00:29:35.073647 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:35.053520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:40855 (local address 127.2.120.1:58305)
0811 00:29:35.065087 (+ 11567us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:35.065111 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:35.065308 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:35.065721 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:35.065726 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:35.065768 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:29:35.066188 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:35.066202 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:35.067709 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:35.067714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:35.070058 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:35.070077 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:35.072008 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:35.072016 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:35.072198 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:35.072234 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 00:29:35.073127 (+ 893us) client_negotiation.cc:240] Negotiation successful
0811 00:29:35.073277 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11392,"thread_start_us":16538,"threads_started":1}
I20260811 00:29:35.075011 17710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea35a40b80e5439e90e7393961f214cf" candidate_uuid: "83b2887cfc784979a7c5887d0019f11c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89a0353c4d4748aea666d5f90978eed4" is_pre_election: true
I20260811 00:29:35.075696 17710 raft_consensus.cc:2454] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83b2887cfc784979a7c5887d0019f11c in term 0.
I20260811 00:29:35.076462 17316 leader_election.cc:302] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83b2887cfc784979a7c5887d0019f11c, 89a0353c4d4748aea666d5f90978eed4; no voters:
I20260811 00:29:35.077030 17796 raft_consensus.cc:2781] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:29:35.077200 17796 raft_consensus.cc:482] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:29:35.077296 17796 raft_consensus.cc:3037] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:29:35.081173 17796 raft_consensus.cc:504] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b2887cfc784979a7c5887d0019f11c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43483 } } peers { permanent_uuid: "303a3053b4ea4c0bb404b81722d3920d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33493 } } peers { permanent_uuid: "89a0353c4d4748aea666d5f90978eed4" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 40855 } }
I20260811 00:29:35.082055 17796 leader_election.cc:288] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [CANDIDATE]: Term 1 election: Requested vote from peers 303a3053b4ea4c0bb404b81722d3920d (127.2.120.2:33493), 89a0353c4d4748aea666d5f90978eed4 (127.2.120.3:40855)
I20260811 00:29:35.083498 17788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:35.052819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:38371 (local address 127.2.120.2:33493)
0811 00:29:35.053041 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:35.053047 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:35.070480 (+ 17433us) server_negotiation.cc:407] Connection header received
0811 00:29:35.070831 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:35.070837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:35.070934 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:35.071007 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:29:35.071928 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:35.073043 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:35.079438 (+ 6395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:35.080887 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:35.081022 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:35.081129 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:35.082778 (+ 1649us) server_negotiation.cc:299] Negotiation successful
0811 00:29:35.082948 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 00:29:35.084381 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:35.054376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:33493 (local address 127.2.120.1:38371)
0811 00:29:35.070367 (+ 15991us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:35.070389 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:35.070659 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:35.071363 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:35.071369 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:35.071398 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:29:35.071737 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:35.071752 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:35.073571 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:35.073576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:35.077431 (+ 3855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:35.077456 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:35.081493 (+ 4037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:35.081501 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:35.081793 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:35.081863 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 00:29:35.083812 (+ 1949us) client_negotiation.cc:240] Negotiation successful
0811 00:29:35.083996 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15490,"thread_start_us":10318,"threads_started":1}
I20260811 00:29:35.085479 17710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea35a40b80e5439e90e7393961f214cf" candidate_uuid: "83b2887cfc784979a7c5887d0019f11c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89a0353c4d4748aea666d5f90978eed4"
I20260811 00:29:35.085774 17710 raft_consensus.cc:3037] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:29:35.086088 17550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea35a40b80e5439e90e7393961f214cf" candidate_uuid: "83b2887cfc784979a7c5887d0019f11c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "303a3053b4ea4c0bb404b81722d3920d"
I20260811 00:29:35.086612 17550 raft_consensus.cc:3037] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:29:35.089287 17710 raft_consensus.cc:2454] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83b2887cfc784979a7c5887d0019f11c in term 1.
I20260811 00:29:35.089934 17316 leader_election.cc:302] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83b2887cfc784979a7c5887d0019f11c, 89a0353c4d4748aea666d5f90978eed4; no voters:
I20260811 00:29:35.090330 17796 raft_consensus.cc:2781] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:29:35.090454 17551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea35a40b80e5439e90e7393961f214cf" candidate_uuid: "83b2887cfc784979a7c5887d0019f11c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "303a3053b4ea4c0bb404b81722d3920d" is_pre_election: true
I20260811 00:29:35.093353 17550 raft_consensus.cc:2454] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83b2887cfc784979a7c5887d0019f11c in term 1.
I20260811 00:29:35.094399 17796 raft_consensus.cc:686] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [term 1 LEADER]: Becoming Leader. State: Replica: 83b2887cfc784979a7c5887d0019f11c, State: Running, Role: LEADER
I20260811 00:29:35.095760 17796 consensus_queue.cc:237] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b2887cfc784979a7c5887d0019f11c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43483 } } peers { permanent_uuid: "303a3053b4ea4c0bb404b81722d3920d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33493 } } peers { permanent_uuid: "89a0353c4d4748aea666d5f90978eed4" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 40855 } }
I20260811 00:29:35.102080 17211 catalog_manager.cc:5495] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c reported cstate change: term changed from 0 to 1, leader changed from <none> to 83b2887cfc784979a7c5887d0019f11c (127.2.120.1). New cstate: current_term: 1 leader_uuid: "83b2887cfc784979a7c5887d0019f11c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83b2887cfc784979a7c5887d0019f11c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "303a3053b4ea4c0bb404b81722d3920d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89a0353c4d4748aea666d5f90978eed4" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 40855 } health_report { overall_health: UNKNOWN } } }
I20260811 00:29:35.151968 17785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:35.145449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60876 (local address 127.2.120.1:43483)
0811 00:29:35.145742 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:35.145752 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:35.145776 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:29:35.145903 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:35.145907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:35.146013 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:35.146204 (+ 191us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:29:35.146663 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:35.147555 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:35.149264 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:35.149763 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:35.150058 (+ 295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:35.150735 (+ 677us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:29:35.151304 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:29:35.151484 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:35.151567 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 00:29:35.151670 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260811 00:29:35.162053 17710 raft_consensus.cc:1261] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4 [term 1 FOLLOWER]: Refusing update from remote peer 83b2887cfc784979a7c5887d0019f11c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:29:35.162074 17550 raft_consensus.cc:1261] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d [term 1 FOLLOWER]: Refusing update from remote peer 83b2887cfc784979a7c5887d0019f11c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:29:35.163426 17806 consensus_queue.cc:1035] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [LEADER]: Connected to new peer: Peer: permanent_uuid: "89a0353c4d4748aea666d5f90978eed4" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 40855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:29:35.164260 17796 consensus_queue.cc:1035] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [LEADER]: Connected to new peer: Peer: permanent_uuid: "303a3053b4ea4c0bb404b81722d3920d" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 00:29:35.179967 17608 tablet.cc:2369] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:29:35.184370 17764 tablet.cc:2369] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:29:35.190544 17441 tablet.cc:2369] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:29:35.196324 17817 mvcc.cc:205] Tried to move back new op lower bound from 7317127885452537856 to 7317127885198946304. Current Snapshot: MvccSnapshot[applied={T|T < 7317127885452537856}]
I20260811 00:29:35.221966 17211 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:29:35.222298 17211 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:29:35.226421 17211 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a0091eca013744d5aac533e58b14dcdc: Sending DeleteTablet for 3 replicas of tablet ea35a40b80e5439e90e7393961f214cf
I20260811 00:29:35.227761 17366 tablet_service.cc:1514] Processing DeleteTablet for tablet ea35a40b80e5439e90e7393961f214cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:29:35 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:60864
I20260811 00:29:35.228246 17529 tablet_service.cc:1514] Processing DeleteTablet for tablet ea35a40b80e5439e90e7393961f214cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:29:35 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:45052
I20260811 00:29:35.229007 17687 tablet_service.cc:1514] Processing DeleteTablet for tablet ea35a40b80e5439e90e7393961f214cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:29:35 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:37774
I20260811 00:29:35.231714 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 17254
I20260811 00:29:35.256567 17827 tablet_replica.cc:330] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c: stopping tablet replica
I20260811 00:29:35.257157 17827 raft_consensus.cc:2229] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:29:35.257854 17827 raft_consensus.cc:2258] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:29:35.260265 17827 ts_tablet_manager.cc:1905] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:29:35.262890 17829 tablet_replica.cc:330] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4: stopping tablet replica
I20260811 00:29:35.263670 17829 raft_consensus.cc:2229] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:29:35.264398 17829 raft_consensus.cc:2258] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:29:35.265163 17828 tablet_replica.cc:330] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d: stopping tablet replica
I20260811 00:29:35.269920 17785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:35.232917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60880 (local address 127.2.120.1:43483)
0811 00:29:35.233165 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:35.233175 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:35.245038 (+ 11863us) server_negotiation.cc:407] Connection header received
0811 00:29:35.245619 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:35.245628 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:35.245737 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:35.245842 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:35.248640 (+ 2798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:35.250215 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:35.253095 (+ 2880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:35.254174 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:35.254314 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:35.260919 (+ 6605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:35.260936 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:35.260955 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:35.261018 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:35.265771 (+ 4753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:35.266875 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:35.266890 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:35.266894 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:35.267138 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:35.267644 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:35.267649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:35.267653 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:35.268141 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:35.268534 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:35.269197 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 00:29:35.269479 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 00:29:35.269925 17828 raft_consensus.cc:2229] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:29:35.270318 17827 ts_tablet_manager.cc:1918] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:29:35.270838 17828 raft_consensus.cc:2258] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:29:35.272104 17435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:35.273490 17829 ts_tablet_manager.cc:1905] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:29:35.273641 17828 ts_tablet_manager.cc:1905] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:29:35.279723 17827 log.cc:1196] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/ea35a40b80e5439e90e7393961f214cf
I20260811 00:29:35.282004 17829 ts_tablet_manager.cc:1918] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:29:35.282320 17829 log.cc:1196] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/ea35a40b80e5439e90e7393961f214cf
I20260811 00:29:35.282979 17829 ts_tablet_manager.cc:1939] T ea35a40b80e5439e90e7393961f214cf P 89a0353c4d4748aea666d5f90978eed4: Deleting consensus metadata
I20260811 00:29:35.284346 17828 ts_tablet_manager.cc:1918] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:29:35.284683 17828 log.cc:1196] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/ea35a40b80e5439e90e7393961f214cf
I20260811 00:29:35.285339 17828 ts_tablet_manager.cc:1939] T ea35a40b80e5439e90e7393961f214cf P 303a3053b4ea4c0bb404b81722d3920d: Deleting consensus metadata
I20260811 00:29:35.285404 17196 catalog_manager.cc:4841] TS 89a0353c4d4748aea666d5f90978eed4 (127.2.120.3:40855): tablet ea35a40b80e5439e90e7393961f214cf (table test-table [id=e4e282968c8f4419bacec192824d00a7]) successfully deleted
I20260811 00:29:35.287626 17195 catalog_manager.cc:4841] TS 303a3053b4ea4c0bb404b81722d3920d (127.2.120.2:33493): tablet ea35a40b80e5439e90e7393961f214cf (table test-table [id=e4e282968c8f4419bacec192824d00a7]) successfully deleted
I20260811 00:29:35.397852 17827 ts_tablet_manager.cc:1939] T ea35a40b80e5439e90e7393961f214cf P 83b2887cfc784979a7c5887d0019f11c: Deleting consensus metadata
I20260811 00:29:35.397855 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 17254
W20260811 00:29:35.423592 17193 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.120.1:43483 (error 108)
I20260811 00:29:35.423790 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 17445
I20260811 00:29:35.428223 17835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:35.425336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43483 (local address 127.0.0.1:60892)
0811 00:29:35.427551 (+ 2215us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:35.427801 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2001,"thread_start_us":1731,"threads_started":1}
W20260811 00:29:35.428884 17193 catalog_manager.cc:4584] TS 83b2887cfc784979a7c5887d0019f11c (127.2.120.1:43483): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ea35a40b80e5439e90e7393961f214cf: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43483: connect: Connection refused (error 111)
I20260811 00:29:35.443262 17788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:35.424782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45066 (local address 127.2.120.2:33493)
0811 00:29:35.424991 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:35.424999 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:35.425085 (+ 86us) server_negotiation.cc:407] Connection header received
0811 00:29:35.425298 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:35.425308 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:35.425376 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:35.425494 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:35.426949 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:35.428782 (+ 1833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:35.430640 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:35.431195 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:35.431386 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:35.434797 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:35.434818 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:35.434841 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:35.434894 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:35.439469 (+ 4575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:35.440362 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:35.440372 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:35.440376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:35.440727 (+ 351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:35.441235 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:35.441240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:35.441244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:35.441891 (+ 647us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:35.442275 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:35.442525 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 00:29:35.442760 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:29:35.444305 17601 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:35.483048 17835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:35.482512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43483 (local address 127.0.0.1:60900)
0811 00:29:35.482730 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:35.482864 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 00:29:35.484316 17835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:35.483848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43483 (local address 127.0.0.1:60914)
0811 00:29:35.484020 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:35.484127 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 00:29:35.545490 17835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:35.544860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43483 (local address 127.0.0.1:60922)
0811 00:29:35.545105 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:35.545206 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 00:29:35.547127 17835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:35.546654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43483 (local address 127.0.0.1:60938)
0811 00:29:35.546864 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:35.546953 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 00:29:35.578162 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 17445
I20260811 00:29:35.603586 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 17613
I20260811 00:29:35.619679 17837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:35.604400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37786 (local address 127.2.120.3:40855)
0811 00:29:35.605122 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:35.605127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:35.605143 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:29:35.605197 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:35.605201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:35.605259 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:35.605392 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:35.607052 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:35.607993 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:35.609353 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:35.609840 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:35.610006 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:35.612840 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:35.612856 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:35.612874 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:35.612909 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:35.616611 (+ 3702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:35.617417 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:35.617424 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:35.617427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:35.617617 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:35.618082 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:35.618085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:35.618088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:35.618508 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:35.618770 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:35.619067 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 00:29:35.619258 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":453,"threads_started":1}
I20260811 00:29:35.620425 17756 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:35.627928 17835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:35.627295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43483 (local address 127.0.0.1:60944)
0811 00:29:35.627581 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:35.627713 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 00:29:35.629269 17835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:35.628753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43483 (local address 127.0.0.1:60948)
0811 00:29:35.628972 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:35.629078 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260811 00:29:35.712441 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 17613
I20260811 00:29:35.735847 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 17174
I20260811 00:29:35.738034 17247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:35.857470 17835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:35.856366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43483 (local address 127.0.0.1:60958)
0811 00:29:35.857052 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:35.857272 (+ 220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 00:29:35.857726 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 17174
I20260811 00:29:35.886828 2528 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 00:29:36.233520 2528 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 00:29:36.233785 2528 mini_ranger.cc:66] Stopping Ranger...
I20260811 00:29:36.591590 2528 mini_ranger.cc:68] Stopped Ranger
2026-08-11 00:29:36.591 UTC [16075] LOG: received smart shutdown request
2026-08-11 00:29:36.595 UTC [16075] LOG: background worker "logical replication launcher" (PID 16084) exited with exit code 1
2026-08-11 00:29:36.596 UTC [16079] LOG: shutting down
2026-08-11 00:29:36.768 UTC [16075] LOG: database system is shut down
2026-08-11T00:29:36Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (62768 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:29:37 dist-test-slave-fd7l krb5kdc[17849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:29:37 dist-test-slave-fd7l krb5kdc[17849](info): set up 2 sockets
Aug 11 00:29:37 dist-test-slave-fd7l krb5kdc[17849](info): commencing operation
krb5kdc: starting...
W20260811 00:29:38.172334 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:29:38 dist-test-slave-fd7l krb5kdc[17849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:29:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:29:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:29:38.472546 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:39805
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:39385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:39805,127.2.120.61:38327,127.2.120.60:33697
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:29:38.856880 17864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:38.857216 17864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:38.857316 17864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:38.865088 17864 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:29:38.865216 17864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:38.865263 17864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:38.865324 17864 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:29:38.865372 17864 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:29:38.876353 17864 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:39385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:39805,127.2.120.61:38327,127.2.120.60:33697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:39805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:38.877933 17864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:38.879359 17864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:38.892900 17870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:38.892947 17873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:38.892879 17871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:38.894003 17864 server_base.cc:1034] running on GCE node
I20260811 00:29:38.894942 17864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:38.896796 17864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:38.898041 17864 hybrid_clock.cc:648] HybridClock initialized: now 1786408178897991 us; error 49 us; skew 500 ppm
I20260811 00:29:38.898478 17864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:38 dist-test-slave-fd7l krb5kdc[17849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408178, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:38.907666 17864 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:29:38.909641 17864 webserver.cc:460] Webserver started at http://127.2.120.62:34333/ using document root <none> and password file <none>
I20260811 00:29:38.910686 17864 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:38.910880 17864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:38.911358 17864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:29:38.914949 17864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "c34805cc9686436b80aaef7cd7790ef3"
format_stamp: "Formatted at 2026-08-11 00:29:38 on dist-test-slave-fd7l"
server_key: "9faf6c03915558d5852202d59e96b1c6"
server_key_iv: "ae60f1175fe716ae35335f1c6feb4860"
server_key_version: "encryptionkey@0"
I20260811 00:29:38.915967 17864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "c34805cc9686436b80aaef7cd7790ef3"
format_stamp: "Formatted at 2026-08-11 00:29:38 on dist-test-slave-fd7l"
server_key: "9faf6c03915558d5852202d59e96b1c6"
server_key_iv: "ae60f1175fe716ae35335f1c6feb4860"
server_key_version: "encryptionkey@0"
I20260811 00:29:38.922070 17864 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 00:29:38.926179 17882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:38.927790 17864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:29:38.928027 17864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "c34805cc9686436b80aaef7cd7790ef3"
format_stamp: "Formatted at 2026-08-11 00:29:38 on dist-test-slave-fd7l"
server_key: "9faf6c03915558d5852202d59e96b1c6"
server_key_iv: "ae60f1175fe716ae35335f1c6feb4860"
server_key_version: "encryptionkey@0"
I20260811 00:29:38.928273 17864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:38.962684 17864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:38.968005 17864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:38.968364 17864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:38.990281 17864 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:39805
I20260811 00:29:38.990314 17941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:39805 every 8 connection(s)
I20260811 00:29:38.991801 17864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:29:38.993479 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 17864
I20260811 00:29:38.993752 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:29:38.994467 2528 external_mini_cluster.cc:1428] Setting key b5854629bb7f72ffaf0828ffb4bc9bec
I20260811 00:29:38.996188 17943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:29:39.003518 17943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.120.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.61@KRBTEST.COM" created.
Aug 11 00:29:39 dist-test-slave-fd7l krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408178, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:29:39.047459 17949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.007347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54138 (local address 127.2.120.62:39805)
0811 00:29:39.009669 (+ 2322us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:39.009683 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:39.009739 (+ 56us) server_negotiation.cc:407] Connection header received
0811 00:29:39.011573 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:39.011585 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:39.011941 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:39.012189 (+ 248us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:39.013706 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:39.017427 (+ 3721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:39.021620 (+ 4193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:39.024134 (+ 2514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:39.024891 (+ 757us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:39.036326 (+ 11435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:39.036359 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:39.036388 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:39.036476 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:39.041928 (+ 5452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:39.042995 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:39.043009 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:39.043016 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:39.043273 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:39.043786 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:39.043794 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:39.043800 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:39.044905 (+ 1105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:39.045158 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:39.045831 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 00:29:39.046287 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":231,"server-negotiator.queue_time_us":2084,"spinlock_wait_cycles":35200,"thread_start_us":1849,"threads_started":1}
I20260811 00:29:39.047540 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.007346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:39805 (local address 127.0.0.1:54138)
0811 00:29:39.008702 (+ 1356us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.008759 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:39.011007 (+ 2248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:39.012244 (+ 1237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:39.012249 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:39.012949 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:39.013348 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:39.013371 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:39.018146 (+ 4775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:39.018159 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:39.020812 (+ 2653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:39.020881 (+ 69us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:39.025049 (+ 4168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:39.025060 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:39.025504 (+ 444us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:39.027297 (+ 1793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:39.027376 (+ 79us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:39.035619 (+ 8243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:39.042201 (+ 6582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:39.042215 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:39.042228 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:39.042783 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:39.043440 (+ 657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:39.043444 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:39.043447 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:39.043647 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:39.045116 (+ 1469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:39.045123 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:39.045356 (+ 233us) client_negotiation.cc:769] Sending connection context
0811 00:29:39.045612 (+ 256us) client_negotiation.cc:240] Negotiation successful
0811 00:29:39.046075 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1010,"mutex_wait_us":2,"thread_start_us":612,"threads_started":1}
I20260811 00:29:39.051168 17943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:29:39.052405 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.051896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:38327 (local address 127.0.0.1:53336)
0811 00:29:39.052137 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.052262 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260811 00:29:39.052762 17888 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.120.61:38327: connect: Connection refused (error 111)
I20260811 00:29:39.054606 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.054174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:38327 (local address 127.0.0.1:53340)
0811 00:29:39.054348 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.054438 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 00:29:39.054992 17943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.61:38327: Network error: Client connection negotiation failed: client connection to 127.2.120.61:38327: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.61.keytab.
Entry for principal kudu/127.2.120.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.61.keytab.
I20260811 00:29:39.073195 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.61:38327
--webserver_interface=127.2.120.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.61.keytab
--principal=kudu/127.2.120.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:39385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:39805,127.2.120.61:38327,127.2.120.60:33697
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:29:39.104598 17943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } attempt: 1
I20260811 00:29:39.106014 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.105454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:38327 (local address 127.0.0.1:53342)
0811 00:29:39.105701 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.105845 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 00:29:39.107218 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.106767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:38327 (local address 127.0.0.1:53352)
0811 00:29:39.106979 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.107052 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260811 00:29:39.107417 17943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.61:38327: Network error: Client connection negotiation failed: client connection to 127.2.120.61:38327: connect: Connection refused (error 111)
I20260811 00:29:39.175676 17943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } attempt: 2
I20260811 00:29:39.176945 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.176430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:38327 (local address 127.0.0.1:53362)
0811 00:29:39.176672 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.176797 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 00:29:39.178225 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.177801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:38327 (local address 127.0.0.1:53364)
0811 00:29:39.177974 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.178074 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 00:29:39.178424 17943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.61:38327: Network error: Client connection negotiation failed: client connection to 127.2.120.61:38327: connect: Connection refused (error 111)
I20260811 00:29:39.269728 17943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } attempt: 3
I20260811 00:29:39.271142 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.270589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:38327 (local address 127.0.0.1:53378)
0811 00:29:39.270848 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.270977 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260811 00:29:39.272403 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.271975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:38327 (local address 127.0.0.1:53390)
0811 00:29:39.272170 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.272249 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 00:29:39.272680 17943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.61:38327: Network error: Client connection negotiation failed: client connection to 127.2.120.61:38327: connect: Connection refused (error 111)
I20260811 00:29:39.415967 17943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } attempt: 4
I20260811 00:29:39.417472 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.416843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:38327 (local address 127.0.0.1:53392)
0811 00:29:39.417206 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.417314 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
I20260811 00:29:39.418784 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.418331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:38327 (local address 127.0.0.1:53400)
0811 00:29:39.418522 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.418641 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 00:29:39.419003 17943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.61:38327: Network error: Client connection negotiation failed: client connection to 127.2.120.61:38327: connect: Connection refused (error 111)
W20260811 00:29:39.461967 17952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:39.462462 17952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:39.462668 17952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:39.473037 17952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:29:39.473245 17952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:39.473341 17952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:39.473443 17952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:29:39.473513 17952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:29:39.485816 17952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:39385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:39805,127.2.120.61:38327,127.2.120.60:33697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.61.keytab
--principal=kudu/127.2.120.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.61:38327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.120.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:39.487412 17952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:39.489114 17952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:39.502871 17958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:39.502902 17957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:39.503556 17960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:39.503957 17952 server_base.cc:1034] running on GCE node
I20260811 00:29:39.504877 17952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:39.506450 17952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:39.507681 17952 hybrid_clock.cc:648] HybridClock initialized: now 1786408179507633 us; error 35 us; skew 500 ppm
I20260811 00:29:39.508072 17952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:39 dist-test-slave-fd7l krb5kdc[17849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408179, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:39.511528 17952 init.cc:377] Logged in from keytab as kudu/127.2.120.61@KRBTEST.COM (short username kudu)
I20260811 00:29:39.513439 17952 webserver.cc:460] Webserver started at http://127.2.120.61:39535/ using document root <none> and password file <none>
I20260811 00:29:39.514236 17952 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:39.514348 17952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:39.514638 17952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:29:39.517976 17952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-1/data/instance:
uuid: "e425446445134bd68ac25b309a551410"
format_stamp: "Formatted at 2026-08-11 00:29:39 on dist-test-slave-fd7l"
server_key: "dc802426056d01e577dc2735ac86fa8a"
server_key_iv: "fc8bc54bce61eee7ba8ec9d6afafb065"
server_key_version: "encryptionkey@0"
I20260811 00:29:39.518826 17952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-1/wal/instance:
uuid: "e425446445134bd68ac25b309a551410"
format_stamp: "Formatted at 2026-08-11 00:29:39 on dist-test-slave-fd7l"
server_key: "dc802426056d01e577dc2735ac86fa8a"
server_key_iv: "fc8bc54bce61eee7ba8ec9d6afafb065"
server_key_version: "encryptionkey@0"
I20260811 00:29:39.530671 17952 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.008s sys 0.003s
I20260811 00:29:39.541705 17967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:39.546785 17952 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.005s sys 0.000s
I20260811 00:29:39.547346 17952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-1/wal
uuid: "e425446445134bd68ac25b309a551410"
format_stamp: "Formatted at 2026-08-11 00:29:39 on dist-test-slave-fd7l"
server_key: "dc802426056d01e577dc2735ac86fa8a"
server_key_iv: "fc8bc54bce61eee7ba8ec9d6afafb065"
server_key_version: "encryptionkey@0"
I20260811 00:29:39.547793 17952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:39.603081 17952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:39.608443 17952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:39.608844 17952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:39.628407 17952 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.61:38327
I20260811 00:29:39.628443 18028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.61:38327 every 8 connection(s)
I20260811 00:29:39.629902 17952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-1/data/info.pb
I20260811 00:29:39.634392 18029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:29:39.637514 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 17952
I20260811 00:29:39.637734 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-1/wal/instance
I20260811 00:29:39.638298 2528 external_mini_cluster.cc:1428] Setting key f6aa0e0c2f472bcf5df60d1f86acd0a0
I20260811 00:29:39.645123 18029 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:29:39 dist-test-slave-fd7l krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408179, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.61@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.120.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.60@KRBTEST.COM" created.
I20260811 00:29:39.673565 18031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.649621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54140 (local address 127.2.120.62:39805)
0811 00:29:39.650201 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:39.650209 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:39.651503 (+ 1294us) server_negotiation.cc:407] Connection header received
0811 00:29:39.654559 (+ 3056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:39.654565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:39.654660 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:39.654789 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:39.657484 (+ 2695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:39.658448 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:39.661316 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:39.661756 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:39.661865 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:39.667076 (+ 5211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:39.667082 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:39.667086 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:39.667136 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:39.670161 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:39.671048 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:39.671060 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:39.671062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:39.671188 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:39.671677 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:39.671681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:39.671683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:39.671993 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:39.672163 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:39.673087 (+ 924us) server_negotiation.cc:299] Negotiation successful
0811 00:29:39.673281 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":288,"threads_started":1}
I20260811 00:29:39.675196 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.650300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:39805 (local address 127.0.0.1:54140)
0811 00:29:39.651243 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.651295 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:39.654077 (+ 2782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:39.655136 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:39.655152 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:39.656131 (+ 979us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:39.657203 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:39.657239 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:39.658673 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:39.658682 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:39.661099 (+ 2417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:39.661126 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:39.662040 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:39.662049 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:39.662448 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:39.663349 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:39.663392 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:39.666740 (+ 3348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:39.670356 (+ 3616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:39.670367 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:39.670400 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:39.670893 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:39.671314 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:39.671320 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:39.671323 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:39.671536 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:39.672149 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:39.672158 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:39.672630 (+ 472us) client_negotiation.cc:769] Sending connection context
0811 00:29:39.672965 (+ 335us) client_negotiation.cc:240] Negotiation successful
0811 00:29:39.673570 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":417,"threads_started":1}
I20260811 00:29:39.677135 18029 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:29:39 dist-test-slave-fd7l krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408179, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.61@KRBTEST.COM for kudu/127.2.120.61@KRBTEST.COM
I20260811 00:29:39.694747 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.677842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:38327 (local address 127.0.0.1:53416)
0811 00:29:39.678059 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.678079 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:39.678215 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:39.679379 (+ 1164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:39.679387 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:39.679932 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:39.680168 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:39.680185 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:39.681646 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:39.681654 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:39.683165 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:39.683180 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:39.684626 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:39.684630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:39.684820 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:39.685494 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:39.685513 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:39.688050 (+ 2537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:39.691690 (+ 3640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:39.691698 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:39.691703 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:39.692089 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:39.692501 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:39.692533 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:39.692545 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:39.692695 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:39.693608 (+ 913us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:39.693619 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:39.693780 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 00:29:39.693935 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 00:29:39.694321 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":3}
I20260811 00:29:39.694844 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.678110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53416 (local address 127.2.120.61:38327)
0811 00:29:39.678916 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:39.678922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:39.678947 (+ 25us) server_negotiation.cc:407] Connection header received
0811 00:29:39.678993 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:39.678999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:39.679190 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:39.679314 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:39.680342 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:39.681491 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:39.683347 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:39.684378 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:39.684596 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:39.688300 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:39.688314 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:39.688331 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:39.688363 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:39.691498 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:39.692236 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:39.692253 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:39.692255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:39.692384 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:39.692815 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:39.692822 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:39.692825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:39.693443 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:39.693708 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:39.694102 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 00:29:39.694448 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":130,"server-negotiator.queue_time_us":638,"thread_start_us":504,"threads_started":1}
I20260811 00:29:39.698612 18029 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:29:39.700146 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.699527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48572)
0811 00:29:39.699771 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.699959 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260811 00:29:39.700459 17971 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
I20260811 00:29:39.702401 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.701990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48582)
0811 00:29:39.702183 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.702274 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 00:29:39.702656 18029 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:33697: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.60.keytab.
Entry for principal kudu/127.2.120.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.60.keytab.
I20260811 00:29:39.708098 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.60:33697
--webserver_interface=127.2.120.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.60.keytab
--principal=kudu/127.2.120.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:39385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:39805,127.2.120.61:38327,127.2.120.60:33697
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:29:39.718266 17943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } attempt: 5
Aug 11 00:29:39 dist-test-slave-fd7l krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408178, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for kudu/127.2.120.61@KRBTEST.COM
I20260811 00:29:39.737443 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.719085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:38327 (local address 127.0.0.1:53428)
0811 00:29:39.719378 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.719407 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:39.719581 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:39.720053 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:39.720057 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:39.720475 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:39.720764 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:39.720773 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:39.722419 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:39.722424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:39.723360 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:39.723376 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:39.724686 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:39.724691 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:39.724869 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:39.725610 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:39.725630 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:39.728252 (+ 2622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:39.733744 (+ 5492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:39.733749 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:39.733753 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:39.734261 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:39.734849 (+ 588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:39.734853 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:39.734856 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:39.735063 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:39.736082 (+ 1019us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:39.736087 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:39.736274 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 00:29:39.736492 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 00:29:39.736873 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260811 00:29:39.737533 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.719385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53428 (local address 127.2.120.61:38327)
0811 00:29:39.719652 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:39.719661 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:39.719688 (+ 27us) server_negotiation.cc:407] Connection header received
0811 00:29:39.719779 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:39.719787 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:39.719882 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:39.720001 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:39.720953 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:39.722215 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:39.723559 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:39.724463 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:39.724681 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:39.728593 (+ 3912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:39.728604 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:39.728612 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:39.728667 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:39.733469 (+ 4802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:39.734434 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:39.734444 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:39.734451 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:39.734661 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:39.735338 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:39.735346 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:39.735352 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:39.735890 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:39.736118 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:39.736706 (+ 588us) server_negotiation.cc:299] Negotiation successful
0811 00:29:39.737048 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 00:29:39.739454 17943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:29:39.741400 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.740736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48590)
0811 00:29:39.741021 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.741169 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 00:29:39.742904 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.742452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48600)
0811 00:29:39.742638 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.742756 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 00:29:39.743319 17943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:33697: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
I20260811 00:29:39.751999 18029 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } attempt: 1
I20260811 00:29:39.753389 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.752867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48616)
0811 00:29:39.753107 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.753222 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 00:29:39.754532 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.754138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48622)
0811 00:29:39.754330 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.754403 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 00:29:39.754773 18029 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:33697: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
I20260811 00:29:39.794636 17943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } attempt: 1
I20260811 00:29:39.796082 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.795456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48636)
0811 00:29:39.795715 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.795843 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 00:29:39.797365 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.796940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48644)
0811 00:29:39.797125 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.797224 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 00:29:39.797639 17943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:33697: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
I20260811 00:29:39.823019 18029 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } attempt: 2
I20260811 00:29:39.824260 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.823771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48660)
0811 00:29:39.823995 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.824098 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 00:29:39.825344 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.824993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48668)
0811 00:29:39.825143 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.825225 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 00:29:39.825600 18029 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:33697: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
I20260811 00:29:39.865957 17943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } attempt: 2
I20260811 00:29:39.867394 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.866719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48682)
0811 00:29:39.867054 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.867176 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 00:29:39.868600 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.868170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48684)
0811 00:29:39.868332 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.868425 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 00:29:39.868860 17943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:33697: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
I20260811 00:29:39.916890 18029 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } attempt: 3
I20260811 00:29:39.918277 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.917766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48692)
0811 00:29:39.918002 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.918095 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 00:29:39.919516 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.919035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48706)
0811 00:29:39.919308 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.919382 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260811 00:29:39.919838 18029 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:33697: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
I20260811 00:29:39.975080 17943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } attempt: 3
I20260811 00:29:39.976672 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.975884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48712)
0811 00:29:39.976241 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.976381 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 00:29:39.977985 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:39.977494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48716)
0811 00:29:39.977712 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:39.977819 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 00:29:39.978518 17943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:33697: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
I20260811 00:29:40.063127 18029 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } attempt: 4
I20260811 00:29:40.064360 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.063856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48718)
0811 00:29:40.064084 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:40.064180 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 00:29:40.065445 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.065096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48724)
0811 00:29:40.065245 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:40.065326 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 00:29:40.065696 18029 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:33697: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
W20260811 00:29:40.107400 18037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:40.107750 18037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:40.107924 18037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:40.115854 18037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:29:40.115988 18037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:40.116110 18037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:40.116202 18037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:29:40.116288 18037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:29:40.127223 18037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:39385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:39805,127.2.120.61:38327,127.2.120.60:33697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.60.keytab
--principal=kudu/127.2.120.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.60:33697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.120.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:40.128808 18037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:40.130247 18037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:40.144204 18042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:40.144356 18043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:40.144272 18046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:40.145656 18037 server_base.cc:1034] running on GCE node
I20260811 00:29:40.146544 18037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:40.148834 18037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:40.150127 18037 hybrid_clock.cc:648] HybridClock initialized: now 1786408180150084 us; error 61 us; skew 500 ppm
I20260811 00:29:40.150583 18037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:40 dist-test-slave-fd7l krb5kdc[17849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408180, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:40.154155 18037 init.cc:377] Logged in from keytab as kudu/127.2.120.60@KRBTEST.COM (short username kudu)
I20260811 00:29:40.155949 18037 webserver.cc:460] Webserver started at http://127.2.120.60:43377/ using document root <none> and password file <none>
I20260811 00:29:40.155959 17943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } attempt: 4
I20260811 00:29:40.157011 18037 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:40.157177 18037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:40.157295 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.156711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48736)
0811 00:29:40.156953 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:40.157083 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 00:29:40.157603 18037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:29:40.158466 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.158093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48742)
0811 00:29:40.158265 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:40.158342 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260811 00:29:40.158771 17943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:33697: Network error: Client connection negotiation failed: client connection to 127.2.120.60:33697: connect: Connection refused (error 111)
I20260811 00:29:40.161552 18037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-2/data/instance:
uuid: "609c98c6732649598c50f26f59c1a9bc"
format_stamp: "Formatted at 2026-08-11 00:29:40 on dist-test-slave-fd7l"
server_key: "c231d54fa55ed7d0e546604ebaf09026"
server_key_iv: "8e29504ffad8b0c51898db115b9fc0af"
server_key_version: "encryptionkey@0"
I20260811 00:29:40.162550 18037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-2/wal/instance:
uuid: "609c98c6732649598c50f26f59c1a9bc"
format_stamp: "Formatted at 2026-08-11 00:29:40 on dist-test-slave-fd7l"
server_key: "c231d54fa55ed7d0e546604ebaf09026"
server_key_iv: "8e29504ffad8b0c51898db115b9fc0af"
server_key_version: "encryptionkey@0"
I20260811 00:29:40.169994 18037 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 00:29:40.175030 18053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:40.178293 18037 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 00:29:40.178750 18037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-2/wal
uuid: "609c98c6732649598c50f26f59c1a9bc"
format_stamp: "Formatted at 2026-08-11 00:29:40 on dist-test-slave-fd7l"
server_key: "c231d54fa55ed7d0e546604ebaf09026"
server_key_iv: "8e29504ffad8b0c51898db115b9fc0af"
server_key_version: "encryptionkey@0"
I20260811 00:29:40.179106 18037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:40.209735 18037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:40.215507 18037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:40.215807 18037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:40.239840 18037 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.60:33697
I20260811 00:29:40.239866 18110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.60:33697 every 8 connection(s)
I20260811 00:29:40.241591 18037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-2/data/info.pb
I20260811 00:29:40.244108 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 18037
I20260811 00:29:40.244372 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407939260739-2528-0/minicluster-data/master-2/wal/instance
I20260811 00:29:40.245033 2528 external_mini_cluster.cc:1428] Setting key e81bff658f74fdfacf6c4a6490daba0c
I20260811 00:29:40.247184 18111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:29:40.257489 18111 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:29:40 dist-test-slave-fd7l krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.60@KRBTEST.COM
Aug 11 00:29:40 dist-test-slave-fd7l krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408180, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.60@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:29:40.274261 18114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.247303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48754 (local address 127.2.120.60:33697)
0811 00:29:40.248253 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:40.248266 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:40.248310 (+ 44us) server_negotiation.cc:407] Connection header received
0811 00:29:40.250119 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:40.250148 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:40.250628 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:40.251156 (+ 528us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:40.252996 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.254517 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.256540 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.257524 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.257762 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:40.263721 (+ 5959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:40.263732 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:40.263744 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:40.263788 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:40.267955 (+ 4167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.269244 (+ 1289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.269252 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.269261 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.269583 (+ 322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.270208 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.270212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.270215 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.270916 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:40.271187 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:40.272299 (+ 1112us) server_negotiation.cc:299] Negotiation successful
0811 00:29:40.273018 (+ 719us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":439,"threads_started":1}
I20260811 00:29:40.280608 18116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.260832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:39805 (local address 127.0.0.1:54152)
0811 00:29:40.262438 (+ 1606us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:40.262497 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:40.262757 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:40.263260 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:40.263269 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:40.264263 (+ 994us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:40.264768 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:40.264783 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.266485 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.266490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:40.267760 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:40.267776 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.268812 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.268822 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:40.269063 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:40.269957 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:40.269984 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:40.273049 (+ 3065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:40.277361 (+ 4312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:40.277370 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:40.277376 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:40.277945 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:40.278504 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:40.278511 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:40.278517 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:40.278725 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:40.279533 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:40.279539 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:40.279733 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 00:29:40.279902 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 00:29:40.280203 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1223,"thread_start_us":977,"threads_started":1}
I20260811 00:29:40.280871 18115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.260888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54152 (local address 127.2.120.62:39805)
0811 00:29:40.261850 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:40.261860 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:40.262656 (+ 796us) server_negotiation.cc:407] Connection header received
0811 00:29:40.262917 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:40.262921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:40.262996 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:40.263098 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:40.264985 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.266260 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.267960 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.268496 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.268715 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:40.273437 (+ 4722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:40.273446 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:40.273451 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:40.273511 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:40.277159 (+ 3648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.278125 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.278131 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.278133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.278308 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.278873 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.278878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.278880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.279369 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:40.279556 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:40.280098 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 00:29:40.280394 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":480,"threads_started":1}
I20260811 00:29:40.283324 18111 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:29:40 dist-test-slave-fd7l krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
Aug 11 00:29:40 dist-test-slave-fd7l krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408180, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.60@KRBTEST.COM for kudu/127.2.120.61@KRBTEST.COM
I20260811 00:29:40.300966 18118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.278161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54168 (local address 127.2.120.62:39805)
0811 00:29:40.278914 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:40.278918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:40.278939 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:29:40.278982 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:40.278985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:40.279043 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:40.279149 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:40.281182 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.282414 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.284617 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.285362 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.285518 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:40.292150 (+ 6632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:40.292160 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:40.292167 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:40.292227 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:40.296642 (+ 4415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.298125 (+ 1483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.298132 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.298138 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.298347 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.298813 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.298818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.298823 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.299249 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:40.299623 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:40.300156 (+ 533us) server_negotiation.cc:299] Negotiation successful
0811 00:29:40.300487 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":311,"threads_started":1}
I20260811 00:29:40.303853 18116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.284198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:38327 (local address 127.0.0.1:53444)
0811 00:29:40.284420 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:40.284443 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:40.284651 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:40.285535 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:40.285541 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:40.286430 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:40.287119 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:40.287144 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.288565 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.288573 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:40.290164 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:40.290185 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.291249 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.291257 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:40.291598 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:40.292669 (+ 1071us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:40.292694 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:40.295230 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:40.300324 (+ 5094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:40.300331 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:40.300336 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:40.301046 (+ 710us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:40.301639 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:40.301644 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:40.301648 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:40.301914 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:40.302746 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:40.302751 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:40.302946 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 00:29:40.303103 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 00:29:40.303470 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 00:29:40.304203 18119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.284381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53444 (local address 127.2.120.61:38327)
0811 00:29:40.285013 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:40.285021 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:40.285043 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:29:40.285110 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:40.285116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:40.285202 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:40.285344 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:40.287400 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.288333 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.290378 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.291020 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.291144 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:40.295619 (+ 4475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:40.295632 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:40.295640 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:40.295827 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:40.300091 (+ 4264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.301301 (+ 1210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.301307 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.301309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.301483 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.302104 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.302109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.302111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.302593 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:40.302741 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:40.303311 (+ 570us) server_negotiation.cc:299] Negotiation successful
0811 00:29:40.303617 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":365,"threads_started":1}
I20260811 00:29:40.305591 18111 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:29:40 dist-test-slave-fd7l krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408180, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.60@KRBTEST.COM for kudu/127.2.120.60@KRBTEST.COM
Aug 11 00:29:40 dist-test-slave-fd7l krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.61@KRBTEST.COM
I20260811 00:29:40.321060 18116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.306459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48768)
0811 00:29:40.306638 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:40.306669 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:40.306789 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:40.307185 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:40.307189 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:40.307610 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:40.307773 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:40.307783 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.308879 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.308888 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:40.309746 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:40.309756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.310479 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.310485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:40.310672 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:40.311305 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:40.311325 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:40.313772 (+ 2447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:40.318107 (+ 4335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:40.318112 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:40.318115 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:40.318516 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:40.318969 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:40.318972 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:40.318974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:40.319161 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:40.319745 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:40.319750 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:40.319939 (+ 189us) client_negotiation.cc:769] Sending connection context
0811 00:29:40.320061 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 00:29:40.320602 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 00:29:40.321249 18114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.306680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48768 (local address 127.2.120.60:33697)
0811 00:29:40.306858 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:40.306864 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:40.306880 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:29:40.306927 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:40.306931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:40.306997 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:40.307104 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:40.307945 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.308753 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.309895 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.310309 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.310441 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:40.313982 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:40.313987 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:40.313990 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:40.314020 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:40.317917 (+ 3897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.318668 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.318676 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.318680 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.318848 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.319295 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.319299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.319301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.319611 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:40.319727 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:40.320308 (+ 581us) server_negotiation.cc:299] Negotiation successful
0811 00:29:40.320805 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":217,"server-negotiator.queue_time_us":60}
I20260811 00:29:40.322849 18120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.303620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53458 (local address 127.2.120.61:38327)
0811 00:29:40.304562 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:40.304568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:40.304595 (+ 27us) server_negotiation.cc:407] Connection header received
0811 00:29:40.304665 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:40.304670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:40.304749 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:40.304845 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:40.306971 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.307968 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.309501 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.309970 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.310142 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:40.315148 (+ 5006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:40.315156 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:40.315163 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:40.315218 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:40.318988 (+ 3770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.319906 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.319911 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.319913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.320186 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.320778 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.320783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.320786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.321168 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:40.321489 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:40.322249 (+ 760us) server_negotiation.cc:299] Negotiation successful
0811 00:29:40.322441 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":771,"thread_start_us":524,"threads_started":1}
I20260811 00:29:40.327947 18111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc: Bootstrap starting.
I20260811 00:29:40.328923 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:29:40.332636 18111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc: Neither blocks nor log segments found. Creating new log.
I20260811 00:29:40.334368 18111 log.cc:824] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc: Log is configured to *not* fsync() on all Append() calls
I20260811 00:29:40.339084 18111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc: No bootstrap required, opened a new log
I20260811 00:29:40.351821 18111 raft_consensus.cc:348] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } }
I20260811 00:29:40.352566 18111 raft_consensus.cc:374] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:29:40.352766 18111 raft_consensus.cc:729] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 609c98c6732649598c50f26f59c1a9bc, State: Initialized, Role: FOLLOWER
I20260811 00:29:40.353739 18111 consensus_queue.cc:260] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } }
I20260811 00:29:40.357406 18118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.335935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54174 (local address 127.2.120.62:39805)
0811 00:29:40.336152 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:40.336159 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:40.336899 (+ 740us) server_negotiation.cc:407] Connection header received
0811 00:29:40.338124 (+ 1225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:40.338131 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:40.338224 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:40.338313 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:40.340903 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.342045 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.344285 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.344997 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.345106 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:40.349292 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:40.349300 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:40.349307 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:40.349354 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:40.353202 (+ 3848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.354109 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.354116 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.354119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.354280 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.354894 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.354902 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.354909 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.355407 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:40.355730 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:40.356526 (+ 796us) server_negotiation.cc:299] Negotiation successful
0811 00:29:40.356879 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 00:29:40.358484 18120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.337066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53468 (local address 127.2.120.61:38327)
0811 00:29:40.337306 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:40.337313 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:40.337994 (+ 681us) server_negotiation.cc:407] Connection header received
0811 00:29:40.338280 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:40.338289 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:40.338385 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:40.338467 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:40.341590 (+ 3123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.343058 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.345747 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.346422 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.346534 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:40.350522 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:40.350529 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:40.350534 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:40.350582 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:40.354534 (+ 3952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.355556 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.355564 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.355568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.355753 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.356308 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.356314 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.356317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.356753 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:40.357062 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:40.357741 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 00:29:40.358067 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 00:29:40.361178 18131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } } }
I20260811 00:29:40.361634 18131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:29:40.363618 18111 sys_catalog.cc:564] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:29:40.364959 18029 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } attempt: 5
I20260811 00:29:40.365335 18114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.338134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48772 (local address 127.2.120.60:33697)
0811 00:29:40.338366 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:40.338372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:40.345286 (+ 6914us) server_negotiation.cc:407] Connection header received
0811 00:29:40.345806 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:40.345811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:40.345900 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:40.345973 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:40.348533 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.349843 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.351923 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.352787 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.352892 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:40.356726 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:40.356734 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:40.356738 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:40.356780 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:40.361052 (+ 4272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.362836 (+ 1784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.362843 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.362847 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.363054 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.363470 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.363475 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.363478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.363919 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:40.364203 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:40.364552 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 00:29:40.364802 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 00:29:40 dist-test-slave-fd7l krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408179, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.61@KRBTEST.COM for kudu/127.2.120.60@KRBTEST.COM
I20260811 00:29:40.379956 18131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:29:40.380375 18131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } }
I20260811 00:29:40.382447 18131 leader_election.cc:288] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c34805cc9686436b80aaef7cd7790ef3 (127.2.120.62:39805), e425446445134bd68ac25b309a551410 (127.2.120.61:38327)
I20260811 00:29:40.387804 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.365701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48774)
0811 00:29:40.365936 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:40.365962 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:40.366111 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:40.366575 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:40.366580 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:40.367158 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:40.367389 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:40.367405 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.368848 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.368857 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:40.370858 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:40.370884 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.371807 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.371814 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:40.372073 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:40.372805 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:40.372833 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:40.376311 (+ 3478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:40.383372 (+ 7061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:40.383383 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:40.383388 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:40.384704 (+ 1316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:40.385344 (+ 640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:40.385349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:40.385354 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:40.385604 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:40.386524 (+ 920us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:40.386529 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:40.386724 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 00:29:40.386935 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 00:29:40.387363 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 00:29:40.389281 18114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.365797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48774 (local address 127.2.120.60:33697)
0811 00:29:40.365992 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:40.365999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:40.366022 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:29:40.366247 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:40.366252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:40.366361 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:40.367559 (+ 1198us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:40.367709 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.368680 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.371063 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.371592 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.371782 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:40.376657 (+ 4875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:40.376668 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:40.376676 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:40.376735 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:40.383065 (+ 6330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.384877 (+ 1812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.384887 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.384893 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.385175 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.385741 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.385750 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.385753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.386354 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:40.386569 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:40.388546 (+ 1977us) server_negotiation.cc:299] Negotiation successful
0811 00:29:40.388843 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 00:29:40.380141 18037 master_runner.cc:186] Error getting master registration for 127.2.120.62:39805: OK, instance_id { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" instance_seqno: 1786408178984610 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 00:29:40.393285 18037 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 00:29:40.393654 18143 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:29:40.393798 18143 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:29:40.397239 18029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410: Bootstrap starting.
I20260811 00:29:40.402521 18029 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410: Neither blocks nor log segments found. Creating new log.
I20260811 00:29:40.405171 18029 log.cc:824] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410: Log is configured to *not* fsync() on all Append() calls
I20260811 00:29:40.411796 18029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410: No bootstrap required, opened a new log
I20260811 00:29:40.425666 18029 raft_consensus.cc:348] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } }
I20260811 00:29:40.426316 18029 raft_consensus.cc:374] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:29:40.426465 18029 raft_consensus.cc:729] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e425446445134bd68ac25b309a551410, State: Initialized, Role: FOLLOWER
I20260811 00:29:40.427325 18029 consensus_queue.cc:260] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } }
I20260811 00:29:40.428763 18147 sys_catalog.cc:455] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } } }
I20260811 00:29:40.429337 18147 sys_catalog.cc:458] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:29:40.430456 18029 sys_catalog.cc:564] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:29:40.430673 18001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "609c98c6732649598c50f26f59c1a9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e425446445134bd68ac25b309a551410" is_pre_election: true
I20260811 00:29:40.431627 18001 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 609c98c6732649598c50f26f59c1a9bc in term 0.
I20260811 00:29:40.432603 18057 leader_election.cc:302] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 609c98c6732649598c50f26f59c1a9bc, e425446445134bd68ac25b309a551410; no voters:
I20260811 00:29:40.433323 18131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:29:40.433521 18131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:29:40.433660 18131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:29:40.436000 17943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } attempt: 5
I20260811 00:29:40.438156 18131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } }
I20260811 00:29:40.440217 18001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "609c98c6732649598c50f26f59c1a9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e425446445134bd68ac25b309a551410"
I20260811 00:29:40.440479 18001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:29:40.444257 18001 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 609c98c6732649598c50f26f59c1a9bc in term 1.
I20260811 00:29:40.444985 18057 leader_election.cc:302] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 609c98c6732649598c50f26f59c1a9bc, e425446445134bd68ac25b309a551410; no voters:
I20260811 00:29:40.445026 18131 leader_election.cc:288] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [CANDIDATE]: Term 1 election: Requested vote from peers c34805cc9686436b80aaef7cd7790ef3 (127.2.120.62:39805), e425446445134bd68ac25b309a551410 (127.2.120.61:38327)
I20260811 00:29:40.445681 18131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:29:40.446216 18131 raft_consensus.cc:686] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [term 1 LEADER]: Becoming Leader. State: Replica: 609c98c6732649598c50f26f59c1a9bc, State: Running, Role: LEADER
I20260811 00:29:40.448186 18131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } }
I20260811 00:29:40.460345 18131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 609c98c6732649598c50f26f59c1a9bc. Latest consensus state: current_term: 1 leader_uuid: "609c98c6732649598c50f26f59c1a9bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } } }
I20260811 00:29:40.460817 18131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [sys.catalog]: This master's current role is: LEADER
Aug 11 00:29:40 dist-test-slave-fd7l krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408178, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for kudu/127.2.120.60@KRBTEST.COM
I20260811 00:29:40.463663 18155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:29:40.467345 18155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260811 00:29:40.468256 18059 tablet.cc:2369] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:29:40.471537 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.436834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:33697 (local address 127.0.0.1:48776)
0811 00:29:40.437132 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:40.437154 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:40.437279 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:40.450533 (+ 13254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:40.450538 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:40.451095 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:40.451320 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:40.451335 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.455707 (+ 4372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.455713 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:40.457229 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:40.457252 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.458507 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.458513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:40.458732 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:40.459543 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:40.459571 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:40.463011 (+ 3440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:40.468160 (+ 5149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:40.468168 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:40.468172 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:40.468821 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:40.469478 (+ 657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:40.469483 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:40.469488 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:40.469700 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:40.470392 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:40.470397 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:40.470570 (+ 173us) client_negotiation.cc:769] Sending connection context
0811 00:29:40.470723 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 00:29:40.471033 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260811 00:29:40.473119 18114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:40.437594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.2.120.60:33697)
0811 00:29:40.449041 (+ 11447us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:40.449049 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:40.449181 (+ 132us) server_negotiation.cc:407] Connection header received
0811 00:29:40.449647 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:40.449661 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:40.450078 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:40.451671 (+ 1593us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:40.452075 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.455468 (+ 3393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.457519 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:40.458310 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:40.463257 (+ 4947us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:40.463505 (+ 248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:40.463511 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:40.463518 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:40.463552 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:40.467960 (+ 4408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.469098 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.469103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.469107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.469317 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:40.469862 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:40.469867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:40.469871 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:40.470240 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:40.472240 (+ 2000us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:40.472387 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 00:29:40.472694 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10873}
I20260811 00:29:40.480180 17943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3: Bootstrap starting.
I20260811 00:29:40.484988 17943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3: Neither blocks nor log segments found. Creating new log.
I20260811 00:29:40.486877 17943 log.cc:824] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3: Log is configured to *not* fsync() on all Append() calls
I20260811 00:29:40.489058 18001 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410 [term 1 FOLLOWER]: Refusing update from remote peer 609c98c6732649598c50f26f59c1a9bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:29:40.491694 18131 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:29:40.503142 17943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3: No bootstrap required, opened a new log
I20260811 00:29:40.516361 18147 sys_catalog.cc:455] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 609c98c6732649598c50f26f59c1a9bc. Latest consensus state: current_term: 1 leader_uuid: "609c98c6732649598c50f26f59c1a9bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } } }
I20260811 00:29:40.516824 18147 sys_catalog.cc:458] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:29:40.520211 18157 mvcc.cc:205] Tried to move back new op lower bound from 7317127907266584576 to 7317127907136995328. Current Snapshot: MvccSnapshot[applied={T|T < 7317127907266584576}]
I20260811 00:29:40.523449 18164 mvcc.cc:205] Tried to move back new op lower bound from 7317127907266584576 to 7317127907136995328. Current Snapshot: MvccSnapshot[applied={T|T < 7317127907266584576}]
I20260811 00:29:40.524547 18153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "609c98c6732649598c50f26f59c1a9bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } } }
I20260811 00:29:40.525054 18153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [sys.catalog]: This master's current role is: LEADER
I20260811 00:29:40.525213 18147 sys_catalog.cc:455] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "609c98c6732649598c50f26f59c1a9bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } } }
I20260811 00:29:40.525490 18147 sys_catalog.cc:458] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:29:40.533469 17943 raft_consensus.cc:348] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } }
I20260811 00:29:40.533993 17943 raft_consensus.cc:374] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:29:40.534130 18155 catalog_manager.cc:1347] Generated new cluster ID: 0e40524a2112495280685cde7e118833
I20260811 00:29:40.534152 17943 raft_consensus.cc:729] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c34805cc9686436b80aaef7cd7790ef3, State: Initialized, Role: FOLLOWER
I20260811 00:29:40.534246 18155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:29:40.535233 17943 consensus_queue.cc:260] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } }
I20260811 00:29:40.538664 18171 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } } }
I20260811 00:29:40.541930 18171 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:29:40.541726 17913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "609c98c6732649598c50f26f59c1a9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c34805cc9686436b80aaef7cd7790ef3"
I20260811 00:29:40.542696 17912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:29:40.541806 17914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "609c98c6732649598c50f26f59c1a9bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c34805cc9686436b80aaef7cd7790ef3" is_pre_election: true
I20260811 00:29:40.541726 17943 sys_catalog.cc:564] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:29:40.556396 17912 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3 [term 1 FOLLOWER]: Refusing update from remote peer 609c98c6732649598c50f26f59c1a9bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:29:40.561110 18153 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:29:40.533648 17952 master_runner.cc:186] Error getting master registration for 127.2.120.62:39805: OK, instance_id { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" instance_seqno: 1786408178984610 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 00:29:40.566216 17952 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 00:29:40.571235 18170 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 00:29:40.571295 18155 catalog_manager.cc:1370] Generated new certificate authority record
W20260811 00:29:40.571506 18170 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:29:40.574577 18155 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:29:40.594430 18155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc: Generated new TSK 0
I20260811 00:29:40.596073 18155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:29:40.607895 18131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "609c98c6732649598c50f26f59c1a9bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } } }
I20260811 00:29:40.607955 18171 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 609c98c6732649598c50f26f59c1a9bc. Latest consensus state: current_term: 1 leader_uuid: "609c98c6732649598c50f26f59c1a9bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } } }
I20260811 00:29:40.608283 18131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 609c98c6732649598c50f26f59c1a9bc [sys.catalog]: This master's current role is: LEADER
I20260811 00:29:40.608309 18171 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:29:40.609226 18181 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "609c98c6732649598c50f26f59c1a9bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c34805cc9686436b80aaef7cd7790ef3" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39805 } } peers { permanent_uuid: "e425446445134bd68ac25b309a551410" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 38327 } } peers { permanent_uuid: "609c98c6732649598c50f26f59c1a9bc" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 33697 } } }
I20260811 00:29:40.609639 18181 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:29:40.628986 18189 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 00:29:41.577579 18170 catalog_manager.cc:1259] Loaded cluster ID: 0e40524a2112495280685cde7e118833
I20260811 00:29:41.577718 18170 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410: loading cluster ID for follower catalog manager: success
I20260811 00:29:41.581152 18170 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410: acquiring CA information for follower catalog manager: success
I20260811 00:29:41.583472 18170 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e425446445134bd68ac25b309a551410: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:29:41.635082 18189 catalog_manager.cc:1259] Loaded cluster ID: 0e40524a2112495280685cde7e118833
I20260811 00:29:41.635226 18189 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3: loading cluster ID for follower catalog manager: success
I20260811 00:29:41.638900 18189 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3: acquiring CA information for follower catalog manager: success
I20260811 00:29:41.641207 18189 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c34805cc9686436b80aaef7cd7790ef3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:29:41.733551 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 17864
I20260811 00:29:41.735057 17937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:41.838526 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 17864
I20260811 00:29:41.863269 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 17952
I20260811 00:29:41.864449 18024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:41.974503 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 17952
I20260811 00:29:42.001219 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 18037
I20260811 00:29:42.002475 18106 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:42.136620 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 18037
2026-08-11T00:29:42Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5233 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:29:42 dist-test-slave-fd7l krb5kdc[18203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:29:42 dist-test-slave-fd7l krb5kdc[18203](info): set up 2 sockets
Aug 11 00:29:42 dist-test-slave-fd7l krb5kdc[18203](info): commencing operation
krb5kdc: starting...
W20260811 00:29:43.362875 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.002s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:29:43 dist-test-slave-fd7l krb5kdc[18203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:29:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:29:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:29:43.676566 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:45085
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:45193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:45085
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:29:44.062779 18217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:44.063165 18217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:44.063478 18217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:44.071568 18217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:29:44.071724 18217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:44.071825 18217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:44.071919 18217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:29:44.071985 18217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:29:44.083185 18217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:45193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:45085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:45085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:44.085085 18217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:44.086853 18217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:44.101176 18228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:44.101210 18225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:44.101279 18224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:44.102300 18217 server_base.cc:1034] running on GCE node
I20260811 00:29:44.103322 18217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:44.105201 18217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:44.106559 18217 hybrid_clock.cc:648] HybridClock initialized: now 1786408184106502 us; error 48 us; skew 500 ppm
I20260811 00:29:44.107026 18217 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:44 dist-test-slave-fd7l krb5kdc[18203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408184, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:44.115020 18217 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:29:44.117059 18217 webserver.cc:460] Webserver started at http://127.2.120.62:35303/ using document root <none> and password file <none>
I20260811 00:29:44.117923 18217 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:44.118101 18217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:44.118485 18217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:29:44.122097 18217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "66262267c2eb4cd0bd45e21e6eb02fe6"
format_stamp: "Formatted at 2026-08-11 00:29:44 on dist-test-slave-fd7l"
server_key: "f66bc4827a1d396a9bbf85a8e3f81970"
server_key_iv: "afaa8768005e58231c0481b7b26e27ce"
server_key_version: "encryptionkey@0"
I20260811 00:29:44.123027 18217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "66262267c2eb4cd0bd45e21e6eb02fe6"
format_stamp: "Formatted at 2026-08-11 00:29:44 on dist-test-slave-fd7l"
server_key: "f66bc4827a1d396a9bbf85a8e3f81970"
server_key_iv: "afaa8768005e58231c0481b7b26e27ce"
server_key_version: "encryptionkey@0"
I20260811 00:29:44.129145 18217 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260811 00:29:44.133284 18237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:44.135078 18217 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:29:44.135330 18217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "66262267c2eb4cd0bd45e21e6eb02fe6"
format_stamp: "Formatted at 2026-08-11 00:29:44 on dist-test-slave-fd7l"
server_key: "f66bc4827a1d396a9bbf85a8e3f81970"
server_key_iv: "afaa8768005e58231c0481b7b26e27ce"
server_key_version: "encryptionkey@0"
I20260811 00:29:44.135596 18217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:44.149494 18217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:44.154392 18217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:44.154736 18217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:44.175797 18217 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:45085
I20260811 00:29:44.175832 18295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:45085 every 8 connection(s)
I20260811 00:29:44.177383 18217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:29:44.178485 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 18217
I20260811 00:29:44.178797 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:29:44.179883 2528 external_mini_cluster.cc:1428] Setting key dc41eea850371340b195af82c9d2335a
I20260811 00:29:44.183205 18296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 00:29:44 dist-test-slave-fd7l krb5kdc[18203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:29:44.198254 18296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6: Bootstrap starting.
I20260811 00:29:44.202298 18296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6: Neither blocks nor log segments found. Creating new log.
I20260811 00:29:44.203743 18296 log.cc:824] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6: Log is configured to *not* fsync() on all Append() calls
I20260811 00:29:44.206638 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:44.182278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50950 (local address 127.2.120.62:45085)
0811 00:29:44.183236 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:44.183254 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:44.183309 (+ 55us) server_negotiation.cc:407] Connection header received
0811 00:29:44.185339 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:44.185367 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:44.185919 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:44.186384 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:44.188049 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:44.189650 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:44.191138 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:44.191954 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:44.192200 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:44.197451 (+ 5251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:44.197460 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:44.197480 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:44.197526 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:44.201213 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:44.202524 (+ 1311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:44.202533 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:44.202542 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:44.202786 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:44.203298 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:44.203304 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:44.203306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:44.204048 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:44.204409 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:44.205075 (+ 666us) server_negotiation.cc:299] Negotiation successful
0811 00:29:44.205571 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":415,"threads_started":1}
I20260811 00:29:44.207231 18296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6: No bootstrap required, opened a new log
I20260811 00:29:44.211383 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:29:44.216288 18296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66262267c2eb4cd0bd45e21e6eb02fe6" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 45085 } }
I20260811 00:29:44.216887 18296 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:29:44.217031 18296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66262267c2eb4cd0bd45e21e6eb02fe6, State: Initialized, Role: FOLLOWER
I20260811 00:29:44.217869 18296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66262267c2eb4cd0bd45e21e6eb02fe6" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 45085 } }
I20260811 00:29:44.218120 18296 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:29:44.218226 18296 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:29:44.218369 18296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:29:44.220683 18296 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66262267c2eb4cd0bd45e21e6eb02fe6" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 45085 } }
I20260811 00:29:44.221189 18296 leader_election.cc:302] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66262267c2eb4cd0bd45e21e6eb02fe6; no voters:
I20260811 00:29:44.221848 18296 leader_election.cc:288] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:29:44.222111 18301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:29:44.223043 18301 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6 [term 1 LEADER]: Becoming Leader. State: Replica: 66262267c2eb4cd0bd45e21e6eb02fe6, State: Running, Role: LEADER
I20260811 00:29:44.223692 18301 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66262267c2eb4cd0bd45e21e6eb02fe6" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 45085 } }
I20260811 00:29:44.224475 18296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:29:44.227592 18303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66262267c2eb4cd0bd45e21e6eb02fe6. Latest consensus state: current_term: 1 leader_uuid: "66262267c2eb4cd0bd45e21e6eb02fe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66262267c2eb4cd0bd45e21e6eb02fe6" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 45085 } } }
I20260811 00:29:44.227366 18302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66262267c2eb4cd0bd45e21e6eb02fe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66262267c2eb4cd0bd45e21e6eb02fe6" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 45085 } } }
I20260811 00:29:44.228298 18302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6 [sys.catalog]: This master's current role is: LEADER
I20260811 00:29:44.228295 18303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6 [sys.catalog]: This master's current role is: LEADER
I20260811 00:29:44.234028 18312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:29:44.239482 18312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:29:44.249645 18312 catalog_manager.cc:1347] Generated new cluster ID: a76d3c1fffd0441d8f799fb774539804
I20260811 00:29:44.249799 18312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:29:44.267690 18312 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:29:44.269300 18312 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:29:44.286448 18312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66262267c2eb4cd0bd45e21e6eb02fe6: Generated new TSK 0
I20260811 00:29:44.287550 18312 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:29:44.327379 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 18217
I20260811 00:29:44.328941 18289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:44.428002 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 18217
2026-08-11T00:29:44Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2287 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:29:44 dist-test-slave-fd7l krb5kdc[18329](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:29:44 dist-test-slave-fd7l krb5kdc[18329](info): set up 2 sockets
Aug 11 00:29:44 dist-test-slave-fd7l krb5kdc[18329](info): commencing operation
krb5kdc: starting...
W20260811 00:29:45.644924 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.005s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:29:45 dist-test-slave-fd7l krb5kdc[18329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:29:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:29:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:29:45.953363 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:42147
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:42147
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:29:46.335309 18344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:46.335736 18344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:46.335929 18344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:46.343611 18344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:29:46.343813 18344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:46.343902 18344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:46.344014 18344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:29:46.344085 18344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:29:46.354844 18344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:42147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:42147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:46.356758 18344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:46.358286 18344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:46.371876 18349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:46.371999 18350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:46.372033 18352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:46.372926 18344 server_base.cc:1034] running on GCE node
I20260811 00:29:46.373862 18344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:46.375519 18344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:46.376775 18344 hybrid_clock.cc:648] HybridClock initialized: now 1786408186376728 us; error 61 us; skew 500 ppm
I20260811 00:29:46.377179 18344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:46 dist-test-slave-fd7l krb5kdc[18329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408186, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:46.380636 18344 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:29:46.382413 18344 webserver.cc:460] Webserver started at http://127.2.120.62:35241/ using document root <none> and password file <none>
I20260811 00:29:46.383204 18344 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:46.383363 18344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:46.383699 18344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:29:46.386989 18344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "edbb106eb3bb48949bd2ebb7b2160f69"
format_stamp: "Formatted at 2026-08-11 00:29:46 on dist-test-slave-fd7l"
server_key: "dc04bb1a3d195776772262af57992d42"
server_key_iv: "ee5d0f725197567b350d9b17232e2e8d"
server_key_version: "encryptionkey@0"
I20260811 00:29:46.387817 18344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "edbb106eb3bb48949bd2ebb7b2160f69"
format_stamp: "Formatted at 2026-08-11 00:29:46 on dist-test-slave-fd7l"
server_key: "dc04bb1a3d195776772262af57992d42"
server_key_iv: "ee5d0f725197567b350d9b17232e2e8d"
server_key_version: "encryptionkey@0"
I20260811 00:29:46.393586 18344 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 00:29:46.397341 18361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:46.398875 18344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:29:46.399101 18344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "edbb106eb3bb48949bd2ebb7b2160f69"
format_stamp: "Formatted at 2026-08-11 00:29:46 on dist-test-slave-fd7l"
server_key: "dc04bb1a3d195776772262af57992d42"
server_key_iv: "ee5d0f725197567b350d9b17232e2e8d"
server_key_version: "encryptionkey@0"
I20260811 00:29:46.399333 18344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:46.425850 18344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:46.430749 18344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:46.431099 18344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:46.451905 18344 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:42147
I20260811 00:29:46.451941 18423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:42147 every 8 connection(s)
I20260811 00:29:46.453588 18344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:29:46.457147 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 18344
I20260811 00:29:46.457386 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:29:46.458290 18424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:29:46.458343 2528 external_mini_cluster.cc:1428] Setting key f62e913017337d5c5d0848857db30768
I20260811 00:29:46.476187 18424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69: Bootstrap starting.
Aug 11 00:29:46 dist-test-slave-fd7l krb5kdc[18329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:29:46.481281 18424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69: Neither blocks nor log segments found. Creating new log.
I20260811 00:29:46.483008 18424 log.cc:824] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69: Log is configured to *not* fsync() on all Append() calls
I20260811 00:29:46.487470 18424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69: No bootstrap required, opened a new log
I20260811 00:29:46.489130 18428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:46.460694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33246 (local address 127.2.120.62:42147)
0811 00:29:46.461559 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:46.461571 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:46.461618 (+ 47us) server_negotiation.cc:407] Connection header received
0811 00:29:46.463524 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:46.463550 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:46.463992 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:46.464356 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:46.466873 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:46.468742 (+ 1869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:46.471018 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:46.471898 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:46.472075 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:46.478124 (+ 6049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:46.478141 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:46.478174 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:46.478267 (+ 93us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:46.483249 (+ 4982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:46.484661 (+ 1412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:46.484670 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:46.484681 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:46.484981 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:46.485472 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:46.485477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:46.485480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:46.486277 (+ 797us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:46.486644 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:46.487111 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 00:29:46.487639 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":386,"threads_started":1}
I20260811 00:29:46.496639 18424 raft_consensus.cc:348] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42147 } }
I20260811 00:29:46.497205 18424 raft_consensus.cc:374] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:29:46.497360 18424 raft_consensus.cc:729] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edbb106eb3bb48949bd2ebb7b2160f69, State: Initialized, Role: FOLLOWER
I20260811 00:29:46.497987 18424 consensus_queue.cc:260] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42147 } }
I20260811 00:29:46.498255 18424 raft_consensus.cc:388] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:29:46.498425 18424 raft_consensus.cc:482] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:29:46.498600 18424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:29:46.501192 18424 raft_consensus.cc:504] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42147 } }
I20260811 00:29:46.501767 18424 leader_election.cc:302] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edbb106eb3bb48949bd2ebb7b2160f69; no voters:
I20260811 00:29:46.502440 18424 leader_election.cc:288] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:29:46.502635 18430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:29:46.503551 18430 raft_consensus.cc:686] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 1 LEADER]: Becoming Leader. State: Replica: edbb106eb3bb48949bd2ebb7b2160f69, State: Running, Role: LEADER
I20260811 00:29:46.504220 18430 consensus_queue.cc:237] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42147 } }
I20260811 00:29:46.505014 18424 sys_catalog.cc:564] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:29:46.507417 18432 sys_catalog.cc:455] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edbb106eb3bb48949bd2ebb7b2160f69. Latest consensus state: current_term: 1 leader_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42147 } } }
I20260811 00:29:46.507510 18431 sys_catalog.cc:455] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42147 } } }
I20260811 00:29:46.508044 18432 sys_catalog.cc:458] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [sys.catalog]: This master's current role is: LEADER
I20260811 00:29:46.510712 18431 sys_catalog.cc:458] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [sys.catalog]: This master's current role is: LEADER
I20260811 00:29:46.510753 18437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:29:46.516475 18437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:29:46.528170 18437 catalog_manager.cc:1347] Generated new cluster ID: 8b1cec063654483dad73038a16521570
I20260811 00:29:46.528342 18437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:29:46.554934 18437 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:29:46.557144 18437 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:29:46.575568 18437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69: Generated new TSK 0
I20260811 00:29:46.577064 18437 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
I20260811 00:29:46.673923 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:42147
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:29:47.098603 18454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:47.098973 18454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:47.099156 18454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:47.107672 18454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:47.107832 18454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:47.108088 18454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:29:47.119199 18454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:42147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:47.120868 18454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:47.122649 18454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:47.139197 18462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:47.139312 18460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:47.139290 18459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:47.140177 18454 server_base.cc:1034] running on GCE node
I20260811 00:29:47.141151 18454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:47.142396 18454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:47.143703 18454 hybrid_clock.cc:648] HybridClock initialized: now 1786408187143638 us; error 97 us; skew 500 ppm
I20260811 00:29:47.144117 18454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:47 dist-test-slave-fd7l krb5kdc[18329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408187, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:47.147989 18454 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:29:47.149879 18454 webserver.cc:460] Webserver started at http://127.2.120.1:33293/ using document root <none> and password file <none>
I20260811 00:29:47.150739 18454 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:47.150923 18454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:47.151274 18454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:29:47.154838 18454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "e621823aae2241c3bc78c2829bc6308c"
format_stamp: "Formatted at 2026-08-11 00:29:47 on dist-test-slave-fd7l"
server_key: "c242391d6240b69ceb669bf0e786ac12"
server_key_iv: "72050bc8694b7e25d868efc798a58a8b"
server_key_version: "encryptionkey@0"
I20260811 00:29:47.155843 18454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "e621823aae2241c3bc78c2829bc6308c"
format_stamp: "Formatted at 2026-08-11 00:29:47 on dist-test-slave-fd7l"
server_key: "c242391d6240b69ceb669bf0e786ac12"
server_key_iv: "72050bc8694b7e25d868efc798a58a8b"
server_key_version: "encryptionkey@0"
I20260811 00:29:47.162253 18454 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 00:29:47.166394 18471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:47.168269 18454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:29:47.168594 18454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "e621823aae2241c3bc78c2829bc6308c"
format_stamp: "Formatted at 2026-08-11 00:29:47 on dist-test-slave-fd7l"
server_key: "c242391d6240b69ceb669bf0e786ac12"
server_key_iv: "72050bc8694b7e25d868efc798a58a8b"
server_key_version: "encryptionkey@0"
I20260811 00:29:47.168864 18454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:47.188956 18454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:47.194082 18454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:47.194470 18454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:47.195641 18454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:29:47.197232 18454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:29:47.197347 18454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:47.197511 18454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:29:47.197587 18454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:47.234046 18454 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:45095
I20260811 00:29:47.234118 18597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:45095 every 8 connection(s)
I20260811 00:29:47.235476 18454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:29:47.245265 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 18454
I20260811 00:29:47.245500 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:29:47.246202 2528 external_mini_cluster.cc:1428] Setting key e8681337486a9cb6c14cb1dacdac8638
Aug 11 00:29:47 dist-test-slave-fd7l krb5kdc[18329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408187, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:29:47.259928 18601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:47.238416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:32985 (local address 127.2.120.62:42147)
0811 00:29:47.238945 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:47.238953 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:47.239934 (+ 981us) server_negotiation.cc:407] Connection header received
0811 00:29:47.241952 (+ 2018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:47.241957 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:47.242048 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:47.242170 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:47.243850 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:47.244761 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:47.246819 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:47.247365 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:47.247514 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:47.252057 (+ 4543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:47.252067 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:47.252075 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:47.252128 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:47.256090 (+ 3962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:47.257179 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:47.257189 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:47.257195 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:47.257373 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:47.257878 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:47.257885 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:47.257891 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:47.258340 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:47.258508 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:47.259357 (+ 849us) server_negotiation.cc:299] Negotiation successful
0811 00:29:47.259658 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":258,"threads_started":1}
I20260811 00:29:47.261454 18602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:47.238970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:32985)
0811 00:29:47.239750 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:47.239793 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:47.241626 (+ 1833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:47.242416 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:47.242424 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:47.242991 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:47.243623 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:47.243642 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:47.244984 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:47.244989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:47.246600 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:47.246625 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:47.247727 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:47.247736 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:47.248066 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:47.248998 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:47.249030 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:47.251748 (+ 2718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:47.256319 (+ 4571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:47.256359 (+ 40us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:47.256376 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:47.257027 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:47.257528 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:47.257532 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:47.257534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:47.257748 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:47.258511 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:47.258519 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:47.258913 (+ 394us) client_negotiation.cc:769] Sending connection context
0811 00:29:47.259184 (+ 271us) client_negotiation.cc:240] Negotiation successful
0811 00:29:47.259791 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":293,"threads_started":1}
I20260811 00:29:47.263353 18599 heartbeater.cc:347] Connected to a master server at 127.2.120.62:42147
I20260811 00:29:47.263724 18599 heartbeater.cc:464] Registering TS with master...
I20260811 00:29:47.264629 18599 heartbeater.cc:511] Master 127.2.120.62:42147 requested a full tablet report, sending...
I20260811 00:29:47.267057 18382 ts_manager.cc:194] Registered new tserver with Master: e621823aae2241c3bc78c2829bc6308c (127.2.120.1:45095)
I20260811 00:29:47.268704 18382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:32985
WARNING: no policy specified for kudu/127.2.120.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
I20260811 00:29:47.314359 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:42147
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:29:47.692821 18607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:47.693164 18607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:47.693325 18607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:47.701161 18607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:47.701304 18607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:47.701509 18607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:29:47.712915 18607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:42147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:47.714447 18607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:47.715947 18607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:47.732491 18613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:47.732414 18617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:47.733125 18615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:47.733793 18607 server_base.cc:1034] running on GCE node
I20260811 00:29:47.734905 18607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:47.736829 18607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:47.738125 18607 hybrid_clock.cc:648] HybridClock initialized: now 1786408187738044 us; error 82 us; skew 500 ppm
I20260811 00:29:47.738497 18607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:47 dist-test-slave-fd7l krb5kdc[18329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408187, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:47.742288 18607 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:29:47.744005 18607 webserver.cc:460] Webserver started at http://127.2.120.2:38579/ using document root <none> and password file <none>
I20260811 00:29:47.744860 18607 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:47.744987 18607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:47.745255 18607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:29:47.748662 18607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "0beb21908b9942079c54e4a8373321a3"
format_stamp: "Formatted at 2026-08-11 00:29:47 on dist-test-slave-fd7l"
server_key: "f0416ee26ea68cd41f825635a26f2b5a"
server_key_iv: "9e80803cb421f9903a49adbaed36f57c"
server_key_version: "encryptionkey@0"
I20260811 00:29:47.749518 18607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "0beb21908b9942079c54e4a8373321a3"
format_stamp: "Formatted at 2026-08-11 00:29:47 on dist-test-slave-fd7l"
server_key: "f0416ee26ea68cd41f825635a26f2b5a"
server_key_iv: "9e80803cb421f9903a49adbaed36f57c"
server_key_version: "encryptionkey@0"
I20260811 00:29:47.755450 18607 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 00:29:47.759485 18624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:47.761296 18607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 00:29:47.761523 18607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "0beb21908b9942079c54e4a8373321a3"
format_stamp: "Formatted at 2026-08-11 00:29:47 on dist-test-slave-fd7l"
server_key: "f0416ee26ea68cd41f825635a26f2b5a"
server_key_iv: "9e80803cb421f9903a49adbaed36f57c"
server_key_version: "encryptionkey@0"
I20260811 00:29:47.761777 18607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:47.777398 18607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:47.782228 18607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:47.782577 18607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:47.783694 18607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:29:47.785542 18607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:29:47.785627 18607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:47.785802 18607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:29:47.785873 18607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:47.822901 18607 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:43507
I20260811 00:29:47.822935 18756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:43507 every 8 connection(s)
I20260811 00:29:47.824254 18607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:29:47.825842 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 18607
I20260811 00:29:47.826032 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:29:47.826565 2528 external_mini_cluster.cc:1428] Setting key da6b44c8448ca6fe35a87c1f88450170
Aug 11 00:29:47 dist-test-slave-fd7l krb5kdc[18329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408187, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:29:47.847510 18760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:47.827284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:56303 (local address 127.2.120.62:42147)
0811 00:29:47.827802 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:47.827809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:47.829077 (+ 1268us) server_negotiation.cc:407] Connection header received
0811 00:29:47.831092 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:47.831099 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:47.831201 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:47.831328 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:47.833135 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:47.833905 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:47.835552 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:47.836041 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:47.836186 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:47.840607 (+ 4421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:47.840613 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:47.840617 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:47.840657 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:47.843527 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:47.844681 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:47.844685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:47.844688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:47.844831 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:47.845470 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:47.845474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:47.845476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:47.845824 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:47.845958 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:47.846987 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0811 00:29:47.847198 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":262,"threads_started":1}
I20260811 00:29:47.849177 18761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:47.827932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:56303)
0811 00:29:47.828896 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:47.828951 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:47.830764 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:47.831607 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:47.831617 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:47.832187 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:47.832909 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:47.832948 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:47.834086 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:47.834092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:47.835384 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:47.835398 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:47.836309 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:47.836317 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:47.836648 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:47.837565 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:47.837590 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:47.840203 (+ 2613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:47.843740 (+ 3537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:47.843755 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:47.843778 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:47.844546 (+ 768us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:47.844979 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:47.844988 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:47.844995 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:47.845289 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:47.845988 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:47.845999 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:47.846526 (+ 527us) client_negotiation.cc:769] Sending connection context
0811 00:29:47.846851 (+ 325us) client_negotiation.cc:240] Negotiation successful
0811 00:29:47.847526 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":359,"threads_started":1}
I20260811 00:29:47.850904 18758 heartbeater.cc:347] Connected to a master server at 127.2.120.62:42147
I20260811 00:29:47.851281 18758 heartbeater.cc:464] Registering TS with master...
I20260811 00:29:47.852312 18758 heartbeater.cc:511] Master 127.2.120.62:42147 requested a full tablet report, sending...
I20260811 00:29:47.854372 18382 ts_manager.cc:194] Registered new tserver with Master: 0beb21908b9942079c54e4a8373321a3 (127.2.120.2:43507)
I20260811 00:29:47.855268 18382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:56303
WARNING: no policy specified for kudu/127.2.120.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
I20260811 00:29:47.892083 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:42147
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:29:48.272167 18599 heartbeater.cc:503] Master 127.2.120.62:42147 was elected leader, sending a full tablet report...
W20260811 00:29:48.279181 18764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:48.279538 18764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:48.279660 18764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:48.288352 18764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:48.288486 18764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:48.288677 18764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:29:48.299283 18764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:42147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:48.300794 18764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:48.302311 18764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:48.318008 18772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:48.318063 18771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:48.318252 18774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:48.319024 18764 server_base.cc:1034] running on GCE node
I20260811 00:29:48.319828 18764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:48.321313 18764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:48.322847 18764 hybrid_clock.cc:648] HybridClock initialized: now 1786408188322788 us; error 65 us; skew 500 ppm
I20260811 00:29:48.323225 18764 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:48 dist-test-slave-fd7l krb5kdc[18329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408188, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:48.327327 18764 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:29:48.329336 18764 webserver.cc:460] Webserver started at http://127.2.120.3:39281/ using document root <none> and password file <none>
I20260811 00:29:48.330189 18764 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:48.330363 18764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:48.330708 18764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:29:48.334106 18764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "77e03eac97a3494189ee753aa206a6f2"
format_stamp: "Formatted at 2026-08-11 00:29:48 on dist-test-slave-fd7l"
server_key: "0adeeaaa05b7824c3eac04daaea7e0ac"
server_key_iv: "90e2254bff1634be3df41351a82396ef"
server_key_version: "encryptionkey@0"
I20260811 00:29:48.335095 18764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "77e03eac97a3494189ee753aa206a6f2"
format_stamp: "Formatted at 2026-08-11 00:29:48 on dist-test-slave-fd7l"
server_key: "0adeeaaa05b7824c3eac04daaea7e0ac"
server_key_iv: "90e2254bff1634be3df41351a82396ef"
server_key_version: "encryptionkey@0"
I20260811 00:29:48.341343 18764 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 00:29:48.345585 18781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:48.347327 18764 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:29:48.347625 18764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "77e03eac97a3494189ee753aa206a6f2"
format_stamp: "Formatted at 2026-08-11 00:29:48 on dist-test-slave-fd7l"
server_key: "0adeeaaa05b7824c3eac04daaea7e0ac"
server_key_iv: "90e2254bff1634be3df41351a82396ef"
server_key_version: "encryptionkey@0"
I20260811 00:29:48.347934 18764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:48.387708 18764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:48.392728 18764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:48.393008 18764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:48.394032 18764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:29:48.395565 18764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:29:48.395691 18764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:48.395845 18764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:29:48.395920 18764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:48.435214 18764 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:43495
I20260811 00:29:48.435243 18910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:43495 every 8 connection(s)
I20260811 00:29:48.436801 18764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:29:48.445710 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 18764
I20260811 00:29:48.445971 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:29:48.446465 2528 external_mini_cluster.cc:1428] Setting key 20f4c0802f9da86614862ef0848dca86
Aug 11 00:29:48 dist-test-slave-fd7l krb5kdc[18329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408188, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:29:48.462803 18913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:48.440923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:45065 (local address 127.2.120.62:42147)
0811 00:29:48.441703 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:48.441711 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:48.443012 (+ 1301us) server_negotiation.cc:407] Connection header received
0811 00:29:48.445412 (+ 2400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:48.445419 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:48.445566 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:48.445715 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:48.447845 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.448842 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.450743 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.451232 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.451394 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:48.455878 (+ 4484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:48.455883 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:48.455887 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:48.455926 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:48.459077 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:48.460047 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:48.460057 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:48.460060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:48.460201 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:48.460776 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:48.460781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:48.460787 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:48.461116 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:48.461310 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:48.462101 (+ 791us) server_negotiation.cc:299] Negotiation successful
0811 00:29:48.462380 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":377,"threads_started":1}
I20260811 00:29:48.463920 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:48.441735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:45065)
0811 00:29:48.442772 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:48.442826 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:48.445021 (+ 2195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:48.446102 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:48.446117 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:48.446860 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:48.447615 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:48.447639 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.449048 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.449053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:48.450518 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:48.450535 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.451535 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.451544 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:48.451818 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:48.452792 (+ 974us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:48.452819 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:48.455595 (+ 2776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:48.459298 (+ 3703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:48.459310 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:48.459340 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:48.459883 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:48.460336 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:48.460341 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:48.460345 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:48.460651 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:48.461260 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:48.461273 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:48.461685 (+ 412us) client_negotiation.cc:769] Sending connection context
0811 00:29:48.461974 (+ 289us) client_negotiation.cc:240] Negotiation successful
0811 00:29:48.462731 (+ 757us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":449,"threads_started":1}
I20260811 00:29:48.465466 18911 heartbeater.cc:347] Connected to a master server at 127.2.120.62:42147
I20260811 00:29:48.465790 18911 heartbeater.cc:464] Registering TS with master...
I20260811 00:29:48.466647 18911 heartbeater.cc:511] Master 127.2.120.62:42147 requested a full tablet report, sending...
I20260811 00:29:48.468674 18382 ts_manager.cc:194] Registered new tserver with Master: 77e03eac97a3494189ee753aa206a6f2 (127.2.120.3:43495)
I20260811 00:29:48.469686 18382 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:45065
I20260811 00:29:48.475486 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:29:48.496405 18913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:48.480948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33262 (local address 127.2.120.62:42147)
0811 00:29:48.481291 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:48.481299 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:48.481625 (+ 326us) server_negotiation.cc:407] Connection header received
0811 00:29:48.481878 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:48.481882 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:48.481974 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:48.482095 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:48.483772 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.484650 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.486816 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.487316 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.487488 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:48.490371 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:48.490377 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:48.490380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:48.490422 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:48.493285 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:48.494065 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:48.494071 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:48.494075 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:48.494233 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:48.494691 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:48.494695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:48.494697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:48.495066 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:48.495355 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:48.495769 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 00:29:48.496046 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260811 00:29:48.501744 18382 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33262:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:29:48.509825 18382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:29:48.554062 18935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:48.529829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42566 (local address 127.2.120.3:43495)
0811 00:29:48.543546 (+ 13717us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:48.543559 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:48.545207 (+ 1648us) server_negotiation.cc:407] Connection header received
0811 00:29:48.545615 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:48.545624 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:48.545861 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:48.546045 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:29:48.547152 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.548255 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.551062 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.552699 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.552849 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:48.553120 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:48.553477 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 00:29:48.553666 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13477,"thread_start_us":13288,"threads_started":1}
I20260811 00:29:48.554191 18931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:48.529357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:43495 (local address 127.0.0.1:42566)
0811 00:29:48.545075 (+ 15718us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:48.545098 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:48.545352 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:48.546429 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:48.546434 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:48.546465 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:29:48.546932 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:48.546944 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.548548 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.548554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:48.550858 (+ 2304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:48.550876 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.552928 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.552937 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:48.553169 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:48.553233 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 00:29:48.553406 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 00:29:48.553645 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15573,"mutex_wait_us":122,"spinlock_wait_cycles":1920,"thread_start_us":5810,"threads_started":1}
I20260811 00:29:48.555335 18933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:48.526079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57774 (local address 127.2.120.1:45095)
0811 00:29:48.542821 (+ 16742us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:48.542836 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:48.544481 (+ 1645us) server_negotiation.cc:407] Connection header received
0811 00:29:48.544817 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:48.544826 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:48.545070 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:48.545285 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:29:48.546382 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.548208 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.551948 (+ 3740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.553803 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.553958 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:48.554207 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:48.554884 (+ 677us) server_negotiation.cc:299] Negotiation successful
0811 00:29:48.555058 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16491,"thread_start_us":16142,"threads_started":1}
I20260811 00:29:48.555570 18934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:48.525352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:43507 (local address 127.0.0.1:47376)
0811 00:29:48.535664 (+ 10312us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:48.540787 (+ 5123us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:48.541149 (+ 362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:48.545262 (+ 4113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:48.545270 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:48.545321 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:29:48.545880 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:48.545894 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.548419 (+ 2525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.548426 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:48.552108 (+ 3682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:48.552126 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.554643 (+ 2517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.554650 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:48.554863 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:48.554908 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 00:29:48.555005 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 00:29:48.555152 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9993,"mutex_wait_us":238,"spinlock_wait_cycles":6016,"thread_start_us":18535,"threads_started":1}
I20260811 00:29:48.556246 18930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:48.525575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:45095 (local address 127.0.0.1:57774)
0811 00:29:48.544280 (+ 18705us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:48.544310 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:48.544551 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:48.545666 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:48.545673 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:48.545702 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:29:48.546199 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:48.546209 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.548408 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.548415 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:48.551653 (+ 3238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:48.551673 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.554388 (+ 2715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.554397 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:48.554605 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:48.554665 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 00:29:48.555611 (+ 946us) client_negotiation.cc:240] Negotiation successful
0811 00:29:48.555939 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18553,"mutex_wait_us":192,"thread_start_us":19113,"threads_started":1}
I20260811 00:29:48.556401 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:48.525570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47376 (local address 127.2.120.2:43507)
0811 00:29:48.544326 (+ 18756us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:48.544333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:48.544363 (+ 30us) server_negotiation.cc:407] Connection header received
0811 00:29:48.544522 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:48.544561 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:48.544930 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:48.546413 (+ 1483us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:29:48.546696 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.548239 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.552489 (+ 4250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.554157 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.554296 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:48.554468 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:48.555834 (+ 1366us) server_negotiation.cc:299] Negotiation successful
0811 00:29:48.556057 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18600,"thread_start_us":18453,"threads_started":1}
I20260811 00:29:48.559259 18835 tablet_service.cc:1467] Processing CreateTablet for tablet 98f4d23a2c894a2e99fc29cb6fd4a496 (DEFAULT_TABLE table=test-table [id=0dd2fb60856743d7b8d10a57cebada3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:29:48.560443 18677 tablet_service.cc:1467] Processing CreateTablet for tablet 98f4d23a2c894a2e99fc29cb6fd4a496 (DEFAULT_TABLE table=test-table [id=0dd2fb60856743d7b8d10a57cebada3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:29:48.561038 18835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98f4d23a2c894a2e99fc29cb6fd4a496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:29:48.561017 18523 tablet_service.cc:1467] Processing CreateTablet for tablet 98f4d23a2c894a2e99fc29cb6fd4a496 (DEFAULT_TABLE table=test-table [id=0dd2fb60856743d7b8d10a57cebada3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:29:48.561688 18677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98f4d23a2c894a2e99fc29cb6fd4a496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:29:48.562484 18523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98f4d23a2c894a2e99fc29cb6fd4a496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:29:48.574306 18936 tablet_bootstrap.cc:492] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3: Bootstrap starting.
I20260811 00:29:48.576862 18937 tablet_bootstrap.cc:492] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2: Bootstrap starting.
I20260811 00:29:48.579123 18938 tablet_bootstrap.cc:492] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c: Bootstrap starting.
I20260811 00:29:48.579558 18936 tablet_bootstrap.cc:654] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3: Neither blocks nor log segments found. Creating new log.
I20260811 00:29:48.580893 18937 tablet_bootstrap.cc:654] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2: Neither blocks nor log segments found. Creating new log.
I20260811 00:29:48.581027 18936 log.cc:824] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3: Log is configured to *not* fsync() on all Append() calls
I20260811 00:29:48.582175 18937 log.cc:824] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:29:48.582505 18938 tablet_bootstrap.cc:654] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c: Neither blocks nor log segments found. Creating new log.
I20260811 00:29:48.583490 18938 log.cc:824] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c: Log is configured to *not* fsync() on all Append() calls
I20260811 00:29:48.584887 18936 tablet_bootstrap.cc:492] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3: No bootstrap required, opened a new log
I20260811 00:29:48.585237 18936 ts_tablet_manager.cc:1397] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260811 00:29:48.586606 18937 tablet_bootstrap.cc:492] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2: No bootstrap required, opened a new log
I20260811 00:29:48.586776 18938 tablet_bootstrap.cc:492] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c: No bootstrap required, opened a new log
I20260811 00:29:48.587073 18938 ts_tablet_manager.cc:1397] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260811 00:29:48.587267 18937 ts_tablet_manager.cc:1397] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 00:29:48.594925 18937 raft_consensus.cc:348] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } }
I20260811 00:29:48.595412 18937 raft_consensus.cc:374] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:29:48.595537 18937 raft_consensus.cc:729] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e03eac97a3494189ee753aa206a6f2, State: Initialized, Role: FOLLOWER
I20260811 00:29:48.596292 18937 consensus_queue.cc:260] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } }
I20260811 00:29:48.597186 18936 raft_consensus.cc:348] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } }
I20260811 00:29:48.597642 18911 heartbeater.cc:503] Master 127.2.120.62:42147 was elected leader, sending a full tablet report...
I20260811 00:29:48.597747 18936 raft_consensus.cc:374] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:29:48.597859 18936 raft_consensus.cc:729] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0beb21908b9942079c54e4a8373321a3, State: Initialized, Role: FOLLOWER
I20260811 00:29:48.598410 18937 ts_tablet_manager.cc:1428] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260811 00:29:48.598793 18936 consensus_queue.cc:260] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } }
I20260811 00:29:48.599020 18938 raft_consensus.cc:348] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } }
I20260811 00:29:48.599606 18938 raft_consensus.cc:374] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:29:48.599757 18938 raft_consensus.cc:729] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621823aae2241c3bc78c2829bc6308c, State: Initialized, Role: FOLLOWER
I20260811 00:29:48.600234 18758 heartbeater.cc:503] Master 127.2.120.62:42147 was elected leader, sending a full tablet report...
I20260811 00:29:48.600541 18938 consensus_queue.cc:260] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } }
I20260811 00:29:48.603193 18936 ts_tablet_manager.cc:1428] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3: Time spent starting tablet: real 0.018s user 0.015s sys 0.001s
I20260811 00:29:48.606158 18938 ts_tablet_manager.cc:1428] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c: Time spent starting tablet: real 0.019s user 0.018s sys 0.000s
W20260811 00:29:48.689172 18912 tablet.cc:2369] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:29:48.715132 18944 raft_consensus.cc:482] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:29:48.715477 18944 raft_consensus.cc:504] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } }
I20260811 00:29:48.717401 18944 leader_election.cc:288] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e621823aae2241c3bc78c2829bc6308c (127.2.120.1:45095), 77e03eac97a3494189ee753aa206a6f2 (127.2.120.3:43495)
I20260811 00:29:48.725214 18933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:48.718043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:58557 (local address 127.2.120.1:45095)
0811 00:29:48.718362 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:48.718370 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:48.718836 (+ 466us) server_negotiation.cc:407] Connection header received
0811 00:29:48.719119 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:48.719124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:48.719188 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:48.719300 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:29:48.719884 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.720680 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.723255 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.724092 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.724270 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:48.724392 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:48.724773 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 00:29:48.724926 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260811 00:29:48.725392 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:48.717755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:45095 (local address 127.2.120.2:58557)
0811 00:29:48.718699 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:48.718716 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:48.718942 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:48.719326 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:48.719331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:48.719360 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:29:48.719718 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:48.719728 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.720839 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.720845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:48.723068 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:48.723084 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.724341 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.724348 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:48.724565 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:48.724610 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 00:29:48.724722 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 00:29:48.724905 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"mutex_wait_us":30,"thread_start_us":526,"threads_started":1}
I20260811 00:29:48.725756 18935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:48.718096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:55107 (local address 127.2.120.3:43495)
0811 00:29:48.718543 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:48.718550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:48.719232 (+ 682us) server_negotiation.cc:407] Connection header received
0811 00:29:48.719452 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:48.719457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:48.719520 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:48.719630 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:29:48.720249 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.721069 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.723659 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.724620 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.724811 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:48.724952 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:48.725321 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 00:29:48.725480 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302}
I20260811 00:29:48.725916 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:48.717830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:43495 (local address 127.2.120.2:55107)
0811 00:29:48.719104 (+ 1274us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:48.719120 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:48.719292 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:48.719654 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:48.719658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:48.719687 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:29:48.720078 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:48.720087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.721224 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.721235 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:48.723465 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:48.723481 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.724907 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.724915 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:48.725123 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:48.725158 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 00:29:48.725277 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 00:29:48.725436 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1159,"mutex_wait_us":175,"thread_start_us":940,"threads_started":1}
I20260811 00:29:48.726563 18549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98f4d23a2c894a2e99fc29cb6fd4a496" candidate_uuid: "0beb21908b9942079c54e4a8373321a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621823aae2241c3bc78c2829bc6308c" is_pre_election: true
I20260811 00:29:48.727033 18858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98f4d23a2c894a2e99fc29cb6fd4a496" candidate_uuid: "0beb21908b9942079c54e4a8373321a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e03eac97a3494189ee753aa206a6f2" is_pre_election: true
I20260811 00:29:48.727159 18549 raft_consensus.cc:2454] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0beb21908b9942079c54e4a8373321a3 in term 0.
I20260811 00:29:48.727543 18858 raft_consensus.cc:2454] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0beb21908b9942079c54e4a8373321a3 in term 0.
I20260811 00:29:48.727985 18629 leader_election.cc:302] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0beb21908b9942079c54e4a8373321a3, e621823aae2241c3bc78c2829bc6308c; no voters:
I20260811 00:29:48.728484 18944 raft_consensus.cc:2781] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:29:48.728677 18944 raft_consensus.cc:482] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:29:48.728796 18944 raft_consensus.cc:3037] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:29:48.731444 18944 raft_consensus.cc:504] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } }
I20260811 00:29:48.732206 18944 leader_election.cc:288] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [CANDIDATE]: Term 1 election: Requested vote from peers e621823aae2241c3bc78c2829bc6308c (127.2.120.1:45095), 77e03eac97a3494189ee753aa206a6f2 (127.2.120.3:43495)
I20260811 00:29:48.732942 18858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98f4d23a2c894a2e99fc29cb6fd4a496" candidate_uuid: "0beb21908b9942079c54e4a8373321a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77e03eac97a3494189ee753aa206a6f2"
I20260811 00:29:48.732944 18549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98f4d23a2c894a2e99fc29cb6fd4a496" candidate_uuid: "0beb21908b9942079c54e4a8373321a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621823aae2241c3bc78c2829bc6308c"
I20260811 00:29:48.733395 18858 raft_consensus.cc:3037] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:29:48.733378 18549 raft_consensus.cc:3037] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:29:48.737049 18549 raft_consensus.cc:2454] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0beb21908b9942079c54e4a8373321a3 in term 1.
I20260811 00:29:48.737049 18858 raft_consensus.cc:2454] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0beb21908b9942079c54e4a8373321a3 in term 1.
I20260811 00:29:48.737659 18629 leader_election.cc:302] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0beb21908b9942079c54e4a8373321a3, 77e03eac97a3494189ee753aa206a6f2, e621823aae2241c3bc78c2829bc6308c; no voters:
I20260811 00:29:48.738163 18944 raft_consensus.cc:2781] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 00:29:48.738607 18600 tablet.cc:2369] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:29:48.738827 18944 raft_consensus.cc:686] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [term 1 LEADER]: Becoming Leader. State: Replica: 0beb21908b9942079c54e4a8373321a3, State: Running, Role: LEADER
I20260811 00:29:48.739687 18944 consensus_queue.cc:237] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } }
I20260811 00:29:48.745463 18382 catalog_manager.cc:5495] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0beb21908b9942079c54e4a8373321a3 (127.2.120.2). New cstate: current_term: 1 leader_uuid: "0beb21908b9942079c54e4a8373321a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } health_report { overall_health: UNKNOWN } } }
I20260811 00:29:48.792761 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:48.785443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47378 (local address 127.2.120.2:43507)
0811 00:29:48.785747 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:48.785760 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:48.785798 (+ 38us) server_negotiation.cc:407] Connection header received
0811 00:29:48.785900 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:48.785903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:48.785986 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:48.786134 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:29:48.786551 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.787323 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.789321 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.789832 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.789963 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:48.790832 (+ 869us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:29:48.791682 (+ 850us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:29:48.791887 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:48.792182 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 00:29:48.792296 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260811 00:29:48.803907 18549 raft_consensus.cc:1261] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c [term 1 FOLLOWER]: Refusing update from remote peer 0beb21908b9942079c54e4a8373321a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:29:48.803944 18858 raft_consensus.cc:1261] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2 [term 1 FOLLOWER]: Refusing update from remote peer 0beb21908b9942079c54e4a8373321a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:29:48.805274 18950 consensus_queue.cc:1035] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:29:48.805760 18944 consensus_queue.cc:1035] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:29:48.821023 18953 mvcc.cc:205] Tried to move back new op lower bound from 7317127941328236544 to 7317127941082869760. Current Snapshot: MvccSnapshot[applied={T|T < 7317127941328236544}]
I20260811 00:29:48.832114 18954 mvcc.cc:205] Tried to move back new op lower bound from 7317127941328236544 to 7317127941082869760. Current Snapshot: MvccSnapshot[applied={T|T < 7317127941328236544}]
I20260811 00:29:48.847980 18382 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:29:48.848376 18382 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:29:48.853662 18382 catalog_manager.cc:5782] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69: Sending DeleteTablet for 3 replicas of tablet 98f4d23a2c894a2e99fc29cb6fd4a496
I20260811 00:29:48.855170 18523 tablet_service.cc:1514] Processing DeleteTablet for tablet 98f4d23a2c894a2e99fc29cb6fd4a496 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:29:48 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:57774
I20260811 00:29:48.855217 18677 tablet_service.cc:1514] Processing DeleteTablet for tablet 98f4d23a2c894a2e99fc29cb6fd4a496 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:29:48 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:47376
I20260811 00:29:48.857980 18835 tablet_service.cc:1514] Processing DeleteTablet for tablet 98f4d23a2c894a2e99fc29cb6fd4a496 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:29:48 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:42566
I20260811 00:29:48.858824 18963 tablet_replica.cc:330] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3: stopping tablet replica
I20260811 00:29:48.859453 18964 tablet_replica.cc:330] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2: stopping tablet replica
I20260811 00:29:48.859793 18963 raft_consensus.cc:2229] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:29:48.859949 18964 raft_consensus.cc:2229] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:29:48.860246 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 18454
I20260811 00:29:48.860713 18964 raft_consensus.cc:2258] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:29:48.861110 18962 tablet_replica.cc:330] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c: stopping tablet replica
I20260811 00:29:48.861616 18962 raft_consensus.cc:2229] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:29:48.862217 18962 raft_consensus.cc:2258] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:29:48.862753 18963 raft_consensus.cc:2258] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:29:48.863585 18964 ts_tablet_manager.cc:1905] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:29:48.865315 18963 ts_tablet_manager.cc:1905] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:29:48.867885 18962 ts_tablet_manager.cc:1905] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:29:48.872628 18964 ts_tablet_manager.cc:1918] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:29:48.872953 18964 log.cc:1196] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/98f4d23a2c894a2e99fc29cb6fd4a496
I20260811 00:29:48.873787 18964 ts_tablet_manager.cc:1939] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 77e03eac97a3494189ee753aa206a6f2: Deleting consensus metadata
I20260811 00:29:48.876724 18362 catalog_manager.cc:4841] TS 77e03eac97a3494189ee753aa206a6f2 (127.2.120.3:43495): tablet 98f4d23a2c894a2e99fc29cb6fd4a496 (table test-table [id=0dd2fb60856743d7b8d10a57cebada3d]) successfully deleted
I20260811 00:29:48.877167 18962 ts_tablet_manager.cc:1918] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:29:48.877506 18962 log.cc:1196] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/98f4d23a2c894a2e99fc29cb6fd4a496
I20260811 00:29:48.878263 18962 ts_tablet_manager.cc:1939] T 98f4d23a2c894a2e99fc29cb6fd4a496 P e621823aae2241c3bc78c2829bc6308c: Deleting consensus metadata
I20260811 00:29:48.880681 18367 catalog_manager.cc:4841] TS e621823aae2241c3bc78c2829bc6308c (127.2.120.1:45095): tablet 98f4d23a2c894a2e99fc29cb6fd4a496 (table test-table [id=0dd2fb60856743d7b8d10a57cebada3d]) successfully deleted
I20260811 00:29:48.884760 18963 ts_tablet_manager.cc:1918] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:29:48.884996 18963 log.cc:1196] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/98f4d23a2c894a2e99fc29cb6fd4a496
I20260811 00:29:48.885605 18963 ts_tablet_manager.cc:1939] T 98f4d23a2c894a2e99fc29cb6fd4a496 P 0beb21908b9942079c54e4a8373321a3: Deleting consensus metadata
Aug 11 00:29:48 dist-test-slave-fd7l krb5kdc[18329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:29:48.888615 18365 catalog_manager.cc:4841] TS 0beb21908b9942079c54e4a8373321a3 (127.2.120.2:43507): tablet 98f4d23a2c894a2e99fc29cb6fd4a496 (table test-table [id=0dd2fb60856743d7b8d10a57cebada3d]) successfully deleted
I20260811 00:29:48.896203 18933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:48.862168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57778 (local address 127.2.120.1:45095)
0811 00:29:48.862371 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:48.862378 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:48.871122 (+ 8744us) server_negotiation.cc:407] Connection header received
0811 00:29:48.871744 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:48.871751 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:48.871851 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:48.871986 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:48.874749 (+ 2763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.876130 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.880039 (+ 3909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:48.880801 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:48.880916 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:48.888115 (+ 7199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:48.888138 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:48.888161 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:48.888217 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:48.892540 (+ 4323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:48.893541 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:48.893555 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:48.893558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:48.893760 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:48.894219 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:48.894224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:48.894227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:48.894774 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:48.895118 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:48.895492 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 00:29:48.895763 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 00:29:48.897091 18593 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:49.011716 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 18454
I20260811 00:29:49.035554 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 18607
Aug 11 00:29:49 dist-test-slave-fd7l krb5kdc[18329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.2@KRBTEST.COM
I20260811 00:29:49.063464 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:49.036432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47390 (local address 127.2.120.2:43507)
0811 00:29:49.036664 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:49.036672 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:49.036701 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:29:49.036952 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:49.036956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:49.037022 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:49.037146 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:49.038510 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:49.039402 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:49.041229 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:49.041755 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:49.041937 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:49.054414 (+ 12477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:49.054443 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:49.054463 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:49.054559 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:49.058850 (+ 4291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:49.060339 (+ 1489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:49.060354 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:49.060357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:49.060687 (+ 330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:49.061254 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:49.061259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:49.061261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:49.061797 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:49.062278 (+ 481us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:49.062676 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 00:29:49.063015 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 00:29:49.064437 18751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:49.191390 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 18607
I20260811 00:29:49.217502 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 18764
Aug 11 00:29:49 dist-test-slave-fd7l krb5kdc[18329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.3@KRBTEST.COM
I20260811 00:29:49.238953 18935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:49.218423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42574 (local address 127.2.120.3:43495)
0811 00:29:49.218645 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:49.218651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:49.218673 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:29:49.218875 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:49.218887 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:49.218957 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:49.219066 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:49.220305 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:49.221113 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:49.222382 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:49.222785 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:49.222927 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:49.232187 (+ 9260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:49.232207 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:49.232226 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:49.232283 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:49.235572 (+ 3289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:49.236553 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:49.236570 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:49.236573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:49.236748 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:49.237177 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:49.237182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:49.237187 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:49.237606 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:49.237917 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:49.238294 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 00:29:49.238548 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 00:29:49.239794 18906 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:49.332947 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 18764
I20260811 00:29:49.355123 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 18344
I20260811 00:29:49.357077 18419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:49.474781 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 18344
I20260811 00:29:49.502954 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:42147
--webserver_interface=127.2.120.62
--webserver_port=35241
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:42147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:29:49.890055 18971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:49.890465 18971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:49.890650 18971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:49.898144 18971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:29:49.898339 18971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:49.898458 18971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:49.898576 18971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:29:49.898669 18971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:29:49.909402 18971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:42147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:42147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=35241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:49.910943 18971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:49.912350 18971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:49.941107 18980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:49.941068 18982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:49.941154 18978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:49.947465 18971 server_base.cc:1034] running on GCE node
I20260811 00:29:49.948766 18971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:49.950630 18971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:49.951990 18971 hybrid_clock.cc:648] HybridClock initialized: now 1786408189951897 us; error 69 us; skew 500 ppm
I20260811 00:29:49.952700 18971 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:49 dist-test-slave-fd7l krb5kdc[18329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408189, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:49.957554 18971 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:29:49.959854 18971 webserver.cc:460] Webserver started at http://127.2.120.62:35241/ using document root <none> and password file <none>
I20260811 00:29:49.960971 18971 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:49.961194 18971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:49.969249 18971 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 00:29:49.973482 18989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:49.975507 18971 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:29:49.975767 18971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "edbb106eb3bb48949bd2ebb7b2160f69"
format_stamp: "Formatted at 2026-08-11 00:29:46 on dist-test-slave-fd7l"
server_key: "dc04bb1a3d195776772262af57992d42"
server_key_iv: "ee5d0f725197567b350d9b17232e2e8d"
server_key_version: "encryptionkey@0"
I20260811 00:29:49.977217 18971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:49.993557 18971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:49.999571 18971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:49.999970 18971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:50.024266 18971 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:42147
I20260811 00:29:50.024286 19049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:42147 every 8 connection(s)
I20260811 00:29:50.026147 18971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:29:50.030136 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 18971
I20260811 00:29:50.031143 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:45095
--local_ip_for_outbound_sockets=127.2.120.1
--tserver_master_addrs=127.2.120.62:42147
--webserver_port=33293
--webserver_interface=127.2.120.1
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:29:50.035697 19051 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:29:50.041615 19051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69: Bootstrap starting.
I20260811 00:29:50.063302 19051 log.cc:824] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69: Log is configured to *not* fsync() on all Append() calls
I20260811 00:29:50.077973 19051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:29:50.078641 19051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69: Bootstrap complete.
I20260811 00:29:50.089406 19051 raft_consensus.cc:348] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42147 } }
I20260811 00:29:50.090404 19051 raft_consensus.cc:729] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: edbb106eb3bb48949bd2ebb7b2160f69, State: Initialized, Role: FOLLOWER
I20260811 00:29:50.091173 19051 consensus_queue.cc:260] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42147 } }
I20260811 00:29:50.091470 19051 raft_consensus.cc:388] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:29:50.091607 19051 raft_consensus.cc:482] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:29:50.091831 19051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 1 FOLLOWER]: Advancing to term 2
I20260811 00:29:50.096500 19051 raft_consensus.cc:504] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42147 } }
I20260811 00:29:50.097265 19051 leader_election.cc:302] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edbb106eb3bb48949bd2ebb7b2160f69; no voters:
I20260811 00:29:50.098021 19051 leader_election.cc:288] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 00:29:50.098183 19055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 00:29:50.100853 19055 raft_consensus.cc:686] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 2 LEADER]: Becoming Leader. State: Replica: edbb106eb3bb48949bd2ebb7b2160f69, State: Running, Role: LEADER
I20260811 00:29:50.101403 19051 sys_catalog.cc:564] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:29:50.101693 19055 consensus_queue.cc:237] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42147 } }
I20260811 00:29:50.105226 19057 sys_catalog.cc:455] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edbb106eb3bb48949bd2ebb7b2160f69. Latest consensus state: current_term: 2 leader_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42147 } } }
I20260811 00:29:50.105737 19057 sys_catalog.cc:458] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [sys.catalog]: This master's current role is: LEADER
I20260811 00:29:50.106739 19065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:29:50.106529 19056 sys_catalog.cc:455] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42147 } } }
I20260811 00:29:50.106880 19056 sys_catalog.cc:458] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [sys.catalog]: This master's current role is: LEADER
I20260811 00:29:50.117803 19065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:29:50.119503 19065 catalog_manager.cc:1259] Loaded cluster ID: 8b1cec063654483dad73038a16521570
I20260811 00:29:50.119710 19065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:29:50.123260 19065 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 00:29:50.142011 19065 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 00:29:50.171149 19065 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:29:50.174337 19065 catalog_manager.cc:5879] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69: Loaded TSK: 0
I20260811 00:29:50.177569 19065 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 00:29:50.473718 19053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:50.474186 19053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:50.474424 19053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:50.488058 19053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:50.488245 19053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:50.488452 19053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:29:50.505517 19053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:45095
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=33293
--tserver_master_addrs=127.2.120.62:42147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:50.507503 19053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:50.509610 19053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:50.527419 19087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:50.527462 19085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:50.528357 19084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:50.530321 19053 server_base.cc:1034] running on GCE node
I20260811 00:29:50.531139 19053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:50.532428 19053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:50.533742 19053 hybrid_clock.cc:648] HybridClock initialized: now 1786408190533668 us; error 68 us; skew 500 ppm
I20260811 00:29:50.534145 19053 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:50 dist-test-slave-fd7l krb5kdc[18329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408190, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:50.538369 19053 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:29:50.540271 19053 webserver.cc:460] Webserver started at http://127.2.120.1:33293/ using document root <none> and password file <none>
I20260811 00:29:50.541247 19053 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:50.541441 19053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:50.548466 19053 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:29:50.551944 19095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:50.553757 19053 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:29:50.554003 19053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "e621823aae2241c3bc78c2829bc6308c"
format_stamp: "Formatted at 2026-08-11 00:29:47 on dist-test-slave-fd7l"
server_key: "c242391d6240b69ceb669bf0e786ac12"
server_key_iv: "72050bc8694b7e25d868efc798a58a8b"
server_key_version: "encryptionkey@0"
I20260811 00:29:50.555153 19053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:50.598780 19053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:50.603853 19053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:50.604192 19053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:50.605474 19053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:29:50.607097 19053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:29:50.607241 19053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:50.607393 19053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:29:50.607489 19053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:50.646145 19053 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:45095
I20260811 00:29:50.646369 19219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:45095 every 8 connection(s)
I20260811 00:29:50.647815 19053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:29:50.649451 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 19053
I20260811 00:29:50.650257 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:43507
--local_ip_for_outbound_sockets=127.2.120.2
--tserver_master_addrs=127.2.120.62:42147
--webserver_port=38579
--webserver_interface=127.2.120.2
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:29:50 dist-test-slave-fd7l krb5kdc[18329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408190, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:29:50.675958 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:50.651975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:34371)
0811 00:29:50.652988 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:50.653035 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:50.654923 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:50.656592 (+ 1669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:50.656607 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:50.657310 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:50.657928 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:50.657949 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:50.659892 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:50.659897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:50.661521 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:50.661539 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:50.662720 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:50.662729 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:50.663055 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:50.663852 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:50.663890 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:50.666422 (+ 2532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:50.670219 (+ 3797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:50.670229 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:50.670245 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:50.670767 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:50.671349 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:50.671355 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:50.671359 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:50.671721 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:50.672983 (+ 1262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:50.672992 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:50.673414 (+ 422us) client_negotiation.cc:769] Sending connection context
0811 00:29:50.673745 (+ 331us) client_negotiation.cc:240] Negotiation successful
0811 00:29:50.674429 (+ 684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":449,"threads_started":1}
I20260811 00:29:50.675935 19222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:50.651579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:34371 (local address 127.2.120.62:42147)
0811 00:29:50.652232 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:50.652244 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:50.653187 (+ 943us) server_negotiation.cc:407] Connection header received
0811 00:29:50.655569 (+ 2382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:50.655581 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:50.655968 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:50.656371 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:50.658189 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:50.659666 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:50.661709 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:50.662437 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:50.662678 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:50.666740 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:50.666749 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:50.666765 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:50.666809 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:50.670017 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:50.670970 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:50.670978 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:50.670984 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:50.671193 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:50.672091 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:50.672095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:50.672098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:50.672809 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:50.673070 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:50.674007 (+ 937us) server_negotiation.cc:299] Negotiation successful
0811 00:29:50.674507 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":325,"threads_started":1}
I20260811 00:29:50.679939 19220 heartbeater.cc:347] Connected to a master server at 127.2.120.62:42147
I20260811 00:29:50.680339 19220 heartbeater.cc:464] Registering TS with master...
I20260811 00:29:50.681473 19220 heartbeater.cc:511] Master 127.2.120.62:42147 requested a full tablet report, sending...
I20260811 00:29:50.684156 19007 ts_manager.cc:194] Registered new tserver with Master: e621823aae2241c3bc78c2829bc6308c (127.2.120.1:45095)
I20260811 00:29:50.685938 19007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:34371
W20260811 00:29:51.028654 19226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:51.029032 19226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:51.029237 19226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:51.037477 19226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:51.037608 19226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:51.037813 19226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:29:51.048460 19226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:43507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=38579
--tserver_master_addrs=127.2.120.62:42147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:51.050063 19226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:51.051712 19226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:51.067329 19231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:51.067379 19232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:51.067488 19235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:51.068830 19226 server_base.cc:1034] running on GCE node
I20260811 00:29:51.069569 19226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:51.070814 19226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:51.072096 19226 hybrid_clock.cc:648] HybridClock initialized: now 1786408191072037 us; error 52 us; skew 500 ppm
I20260811 00:29:51.072489 19226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:51 dist-test-slave-fd7l krb5kdc[18329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408191, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:51.076256 19226 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:29:51.078132 19226 webserver.cc:460] Webserver started at http://127.2.120.2:38579/ using document root <none> and password file <none>
I20260811 00:29:51.078933 19226 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:51.079078 19226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:51.086051 19226 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:29:51.089393 19245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:51.091293 19226 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:29:51.091548 19226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "0beb21908b9942079c54e4a8373321a3"
format_stamp: "Formatted at 2026-08-11 00:29:47 on dist-test-slave-fd7l"
server_key: "f0416ee26ea68cd41f825635a26f2b5a"
server_key_iv: "9e80803cb421f9903a49adbaed36f57c"
server_key_version: "encryptionkey@0"
I20260811 00:29:51.092857 19226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:51.119673 19226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:51.124639 19226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:51.124939 19226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:51.126098 19226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:29:51.127480 19226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:29:51.127606 19226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:51.127735 19226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:29:51.127811 19226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:51.164101 19226 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:43507
I20260811 00:29:51.164213 19373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:43507 every 8 connection(s)
I20260811 00:29:51.165673 19226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:29:51.172827 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 19226
I20260811 00:29:51.173540 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:43495
--local_ip_for_outbound_sockets=127.2.120.3
--tserver_master_addrs=127.2.120.62:42147
--webserver_port=39281
--webserver_interface=127.2.120.3
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:29:51 dist-test-slave-fd7l krb5kdc[18329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408191, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:29:51.191335 19222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:51.168783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:38305 (local address 127.2.120.62:42147)
0811 00:29:51.168978 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:51.168982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:51.170312 (+ 1330us) server_negotiation.cc:407] Connection header received
0811 00:29:51.172314 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:51.172322 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:51.172625 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:51.172784 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:51.174829 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.175599 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.177681 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.178177 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.178352 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:51.182999 (+ 4647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:51.183011 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:51.183019 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:51.183088 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:51.187218 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:51.188197 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:51.188207 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:51.188214 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:51.188435 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:51.189024 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:51.189033 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:51.189039 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:51.189528 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:51.189692 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:51.190493 (+ 801us) server_negotiation.cc:299] Negotiation successful
0811 00:29:51.190799 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 00:29:51.191937 19376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:51.169272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:38305)
0811 00:29:51.170089 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:51.170130 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:51.172004 (+ 1874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:51.173159 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:51.173171 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:51.173887 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:51.174587 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:51.174610 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.175798 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.175813 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:51.177486 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:51.177505 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.178472 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.178483 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:51.178769 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:51.179641 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:51.179670 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:51.182663 (+ 2993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:51.187452 (+ 4789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:51.187475 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:51.187491 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:51.188008 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:51.188643 (+ 635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:51.188647 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:51.188649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:51.188854 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:51.189690 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:51.189697 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:51.190064 (+ 367us) client_negotiation.cc:769] Sending connection context
0811 00:29:51.190309 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 00:29:51.190745 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":333,"threads_started":1}
I20260811 00:29:51.193784 19374 heartbeater.cc:347] Connected to a master server at 127.2.120.62:42147
I20260811 00:29:51.194123 19374 heartbeater.cc:464] Registering TS with master...
I20260811 00:29:51.194896 19374 heartbeater.cc:511] Master 127.2.120.62:42147 requested a full tablet report, sending...
I20260811 00:29:51.196744 19007 ts_manager.cc:194] Registered new tserver with Master: 0beb21908b9942079c54e4a8373321a3 (127.2.120.2:43507)
I20260811 00:29:51.197757 19007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:38305
W20260811 00:29:51.545907 19378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:51.546314 19378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:51.546495 19378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:51.554808 19378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:51.555032 19378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:51.555233 19378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:29:51.565984 19378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:43495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=39281
--tserver_master_addrs=127.2.120.62:42147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:51.567549 19378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:51.569114 19378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:51.584395 19385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:51.584349 19383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:51.585021 19387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:51.585481 19378 server_base.cc:1034] running on GCE node
I20260811 00:29:51.586462 19378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:51.587641 19378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:51.588887 19378 hybrid_clock.cc:648] HybridClock initialized: now 1786408191588822 us; error 54 us; skew 500 ppm
I20260811 00:29:51.589249 19378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:51 dist-test-slave-fd7l krb5kdc[18329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408191, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:51.592847 19378 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:29:51.594630 19378 webserver.cc:460] Webserver started at http://127.2.120.3:39281/ using document root <none> and password file <none>
I20260811 00:29:51.595387 19378 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:51.595530 19378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:51.602155 19378 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 00:29:51.606243 19396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:51.607987 19378 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:29:51.608237 19378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "77e03eac97a3494189ee753aa206a6f2"
format_stamp: "Formatted at 2026-08-11 00:29:48 on dist-test-slave-fd7l"
server_key: "0adeeaaa05b7824c3eac04daaea7e0ac"
server_key_iv: "90e2254bff1634be3df41351a82396ef"
server_key_version: "encryptionkey@0"
I20260811 00:29:51.609457 19378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:51.628414 19378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:51.632980 19378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:51.633469 19378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:51.634856 19378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:29:51.636579 19378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:29:51.636718 19378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:51.636862 19378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:29:51.636940 19378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:51.674335 19378 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:43495
I20260811 00:29:51.674393 19523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:43495 every 8 connection(s)
I20260811 00:29:51.675945 19378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:29:51.678088 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 19378
I20260811 00:29:51.689811 19220 heartbeater.cc:503] Master 127.2.120.62:42147 was elected leader, sending a full tablet report...
Aug 11 00:29:51 dist-test-slave-fd7l krb5kdc[18329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408191, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:29:51.701200 19222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:51.679256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33274 (local address 127.2.120.62:42147)
0811 00:29:51.679450 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:51.679458 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:51.685204 (+ 5746us) server_negotiation.cc:407] Connection header received
0811 00:29:51.685428 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:51.685433 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:51.685515 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:51.685642 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:51.687782 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.689015 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.690974 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.691691 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.691821 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:51.694881 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:51.694885 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:51.694889 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:51.694923 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:51.698117 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:51.699085 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:51.699090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:51.699092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:51.699242 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:51.699701 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:51.699704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:51.699706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:51.699998 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:51.700203 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:51.700699 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 00:29:51.700877 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 00:29:51.703552 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:51.679256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:52483 (local address 127.2.120.62:42147)
0811 00:29:51.685598 (+ 6342us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:51.685602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:51.685618 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:29:51.685666 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:51.685669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:51.685738 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:51.685825 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:51.688118 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.689246 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.691477 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.692205 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.692354 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:51.696129 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:51.696134 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:51.696137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:51.696174 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:51.699665 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:51.700753 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:51.700758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:51.700761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:51.700910 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:51.701544 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:51.701550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:51.701553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:51.701863 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:51.702081 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:51.702873 (+ 792us) server_negotiation.cc:299] Negotiation successful
0811 00:29:51.703118 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":6247,"thread_start_us":5989,"threads_started":1}
I20260811 00:29:51.704820 19528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:51.679993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:52483)
0811 00:29:51.681161 (+ 1168us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:51.681223 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:51.684622 (+ 3399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:51.686199 (+ 1577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:51.686216 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:51.687062 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:29:51.687915 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:51.687938 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.689417 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.689423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:51.691307 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:51.691321 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.692486 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.692497 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:51.692863 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:51.693509 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:29:51.693532 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:29:51.695868 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:29:51.699838 (+ 3970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:51.699848 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:51.699874 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:51.700599 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:51.701074 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:29:51.701078 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:29:51.701081 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:29:51.701380 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:29:51.702071 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:29:51.702079 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:29:51.702487 (+ 408us) client_negotiation.cc:769] Sending connection context
0811 00:29:51.702774 (+ 287us) client_negotiation.cc:240] Negotiation successful
0811 00:29:51.703364 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":500,"threads_started":1}
I20260811 00:29:51.706123 19524 heartbeater.cc:347] Connected to a master server at 127.2.120.62:42147
I20260811 00:29:51.706437 19524 heartbeater.cc:464] Registering TS with master...
I20260811 00:29:51.707175 19524 heartbeater.cc:511] Master 127.2.120.62:42147 requested a full tablet report, sending...
I20260811 00:29:51.708730 19007 ts_manager.cc:194] Registered new tserver with Master: 77e03eac97a3494189ee753aa206a6f2 (127.2.120.3:43495)
I20260811 00:29:51.709566 19007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:52483
I20260811 00:29:51.716922 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:29:51.724669 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:51.717910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33288 (local address 127.2.120.62:42147)
0811 00:29:51.718200 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:51.718211 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:51.718596 (+ 385us) server_negotiation.cc:407] Connection header received
0811 00:29:51.718797 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:51.718801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:51.718983 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:51.719128 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:29:51.719623 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.720427 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.722052 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.722557 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.722675 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:51.723284 (+ 609us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:29:51.723910 (+ 626us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:29:51.724047 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:51.724194 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 00:29:51.724277 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 00:29:51.725865 19007 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:33288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:29:51.734215 19007 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:29:51.763911 19537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:51.750076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42578 (local address 127.2.120.3:43495)
0811 00:29:51.750907 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:51.750918 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:51.754388 (+ 3470us) server_negotiation.cc:407] Connection header received
0811 00:29:51.754795 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:51.754801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:51.755003 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:51.755212 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:29:51.756004 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.757606 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.760647 (+ 3041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.762230 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.762398 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:51.762658 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:51.763309 (+ 651us) server_negotiation.cc:299] Negotiation successful
0811 00:29:51.763516 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":504,"threads_started":1}
I20260811 00:29:51.763916 19540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:51.750996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:43495 (local address 127.0.0.1:42578)
0811 00:29:51.754036 (+ 3040us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:51.754060 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:51.754560 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:51.755244 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:51.755249 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:51.755277 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:29:51.755781 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:51.755796 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.757756 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.757762 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:51.760384 (+ 2622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:51.760401 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.762893 (+ 2492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.762901 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:51.763080 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:51.763129 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 00:29:51.763241 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 00:29:51.763406 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2899,"mutex_wait_us":3,"spinlock_wait_cycles":1792,"thread_start_us":2760,"threads_started":1}
I20260811 00:29:51.763972 19536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:51.748898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47404 (local address 127.2.120.2:43507)
0811 00:29:51.749889 (+ 991us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:51.749900 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:51.753040 (+ 3140us) server_negotiation.cc:407] Connection header received
0811 00:29:51.753170 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:51.753176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:51.753465 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:51.753692 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:29:51.755525 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.757018 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.760381 (+ 3363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.761823 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.762027 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:51.762307 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:51.763452 (+ 1145us) server_negotiation.cc:299] Negotiation successful
0811 00:29:51.763655 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":818,"thread_start_us":663,"threads_started":1}
I20260811 00:29:51.764256 19539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:51.749357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:45095 (local address 127.0.0.1:57782)
0811 00:29:51.753194 (+ 3837us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:51.753224 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:51.753431 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:51.754163 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:51.754169 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:51.754222 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:29:51.754782 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:51.754797 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.757619 (+ 2822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.757626 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:51.760306 (+ 2680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:51.760324 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.762573 (+ 2249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.762579 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:51.762762 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:51.762796 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 00:29:51.762908 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 00:29:51.763022 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3632,"mutex_wait_us":3,"spinlock_wait_cycles":3456,"thread_start_us":3398,"threads_started":1}
I20260811 00:29:51.764667 19535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:51.748613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:43507 (local address 127.0.0.1:47404)
0811 00:29:51.749510 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:51.749581 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:51.752897 (+ 3316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:51.754028 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:51.754038 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:51.754089 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:29:51.755253 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:51.755266 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.757169 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.757178 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:51.760133 (+ 2955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:51.760160 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.762185 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.762196 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:51.762450 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:51.762527 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 00:29:51.762691 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 00:29:51.762942 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":702,"thread_start_us":345,"threads_started":1}
I20260811 00:29:51.765009 19538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:51.750123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57782 (local address 127.2.120.1:45095)
0811 00:29:51.751361 (+ 1238us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:51.751387 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:51.753364 (+ 1977us) server_negotiation.cc:407] Connection header received
0811 00:29:51.753670 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:51.753677 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:51.753896 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:51.754134 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:29:51.755756 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.757450 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.760540 (+ 3090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.762230 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.762464 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:51.762688 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:51.764411 (+ 1723us) server_negotiation.cc:299] Negotiation successful
0811 00:29:51.764659 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1010,"thread_start_us":784,"threads_started":1}
I20260811 00:29:51.768846 19451 tablet_service.cc:1467] Processing CreateTablet for tablet 8d7bc2f3a59b422bb1a9d8afa5c3ffae (DEFAULT_TABLE table=test-table [id=4fd6f1c6247b409d86929ece884237cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:29:51.768846 19302 tablet_service.cc:1467] Processing CreateTablet for tablet 8d7bc2f3a59b422bb1a9d8afa5c3ffae (DEFAULT_TABLE table=test-table [id=4fd6f1c6247b409d86929ece884237cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:29:51.768844 19149 tablet_service.cc:1467] Processing CreateTablet for tablet 8d7bc2f3a59b422bb1a9d8afa5c3ffae (DEFAULT_TABLE table=test-table [id=4fd6f1c6247b409d86929ece884237cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:29:51.770555 19302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d7bc2f3a59b422bb1a9d8afa5c3ffae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:29:51.770553 19451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d7bc2f3a59b422bb1a9d8afa5c3ffae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:29:51.770654 19149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d7bc2f3a59b422bb1a9d8afa5c3ffae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:29:51.785694 19541 tablet_bootstrap.cc:492] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c: Bootstrap starting.
I20260811 00:29:51.789597 19543 tablet_bootstrap.cc:492] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2: Bootstrap starting.
I20260811 00:29:51.789927 19541 tablet_bootstrap.cc:654] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c: Neither blocks nor log segments found. Creating new log.
I20260811 00:29:51.792276 19541 log.cc:824] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c: Log is configured to *not* fsync() on all Append() calls
I20260811 00:29:51.793511 19542 tablet_bootstrap.cc:492] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3: Bootstrap starting.
I20260811 00:29:51.793610 19543 tablet_bootstrap.cc:654] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2: Neither blocks nor log segments found. Creating new log.
I20260811 00:29:51.794948 19543 log.cc:824] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:29:51.796552 19541 tablet_bootstrap.cc:492] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c: No bootstrap required, opened a new log
I20260811 00:29:51.796912 19541 ts_tablet_manager.cc:1397] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260811 00:29:51.797959 19543 tablet_bootstrap.cc:492] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2: No bootstrap required, opened a new log
I20260811 00:29:51.798255 19543 ts_tablet_manager.cc:1397] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260811 00:29:51.798679 19542 tablet_bootstrap.cc:654] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3: Neither blocks nor log segments found. Creating new log.
I20260811 00:29:51.800160 19542 log.cc:824] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3: Log is configured to *not* fsync() on all Append() calls
I20260811 00:29:51.804116 19542 tablet_bootstrap.cc:492] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3: No bootstrap required, opened a new log
I20260811 00:29:51.804466 19542 ts_tablet_manager.cc:1397] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260811 00:29:51.807276 19543 raft_consensus.cc:348] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } }
I20260811 00:29:51.807829 19543 raft_consensus.cc:374] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:29:51.807941 19543 raft_consensus.cc:729] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e03eac97a3494189ee753aa206a6f2, State: Initialized, Role: FOLLOWER
I20260811 00:29:51.808671 19543 consensus_queue.cc:260] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } }
I20260811 00:29:51.809054 19541 raft_consensus.cc:348] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } }
I20260811 00:29:51.809547 19541 raft_consensus.cc:374] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:29:51.809672 19541 raft_consensus.cc:729] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e621823aae2241c3bc78c2829bc6308c, State: Initialized, Role: FOLLOWER
I20260811 00:29:51.810002 19524 heartbeater.cc:503] Master 127.2.120.62:42147 was elected leader, sending a full tablet report...
I20260811 00:29:51.810374 19541 consensus_queue.cc:260] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } }
I20260811 00:29:51.810627 19543 ts_tablet_manager.cc:1428] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260811 00:29:51.818627 19542 raft_consensus.cc:348] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } }
I20260811 00:29:51.819438 19542 raft_consensus.cc:374] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:29:51.819597 19542 raft_consensus.cc:729] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0beb21908b9942079c54e4a8373321a3, State: Initialized, Role: FOLLOWER
I20260811 00:29:51.820639 19542 consensus_queue.cc:260] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } }
I20260811 00:29:51.822633 19541 ts_tablet_manager.cc:1428] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c: Time spent starting tablet: real 0.026s user 0.011s sys 0.013s
I20260811 00:29:51.823380 19374 heartbeater.cc:503] Master 127.2.120.62:42147 was elected leader, sending a full tablet report...
I20260811 00:29:51.824115 19542 ts_tablet_manager.cc:1428] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3: Time spent starting tablet: real 0.019s user 0.018s sys 0.000s
I20260811 00:29:51.891511 19547 raft_consensus.cc:482] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:29:51.891878 19547 raft_consensus.cc:504] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } }
I20260811 00:29:51.893778 19547 leader_election.cc:288] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0beb21908b9942079c54e4a8373321a3 (127.2.120.2:43507), e621823aae2241c3bc78c2829bc6308c (127.2.120.1:45095)
W20260811 00:29:51.900799 19221 tablet.cc:2369] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:29:51.901598 19528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:51.894092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:45095 (local address 127.2.120.3:59687)
0811 00:29:51.894334 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:51.894355 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:51.894499 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:51.895051 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:51.895058 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:51.895105 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:29:51.895348 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:51.895362 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.896778 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.896786 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:51.898589 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:51.898604 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.900551 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.900561 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:51.900773 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:51.900813 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 00:29:51.900935 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 00:29:51.901256 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 00:29:51.901665 19538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:51.894419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:59687 (local address 127.2.120.1:45095)
0811 00:29:51.894646 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:51.894652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:51.894676 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:29:51.894773 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:51.894779 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:51.894873 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:51.894979 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:29:51.895550 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.896642 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.898796 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.900259 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.900417 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:51.900619 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:51.901000 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 00:29:51.901278 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 00:29:51.902771 19536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:51.894498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:56915 (local address 127.2.120.2:43507)
0811 00:29:51.894759 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:51.894769 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:51.895359 (+ 590us) server_negotiation.cc:407] Connection header received
0811 00:29:51.895457 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:51.895466 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:51.895567 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:51.895646 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:29:51.896398 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.897602 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.900148 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.901444 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.901601 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:51.901729 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:51.902349 (+ 620us) server_negotiation.cc:299] Negotiation successful
0811 00:29:51.902502 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 00:29:51.902815 19550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:51.894347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:43507 (local address 127.2.120.3:56915)
0811 00:29:51.895241 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:51.895256 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 00:29:51.895358 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:29:51.895693 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:29:51.895697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:29:51.895721 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:29:51.896115 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:51.896128 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.897830 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.897835 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:51.899896 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:29:51.899917 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.901916 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.901924 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:29:51.902119 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:51.902176 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 00:29:51.902311 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 00:29:51.902475 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"thread_start_us":511,"threads_started":1}
I20260811 00:29:51.903694 19170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d7bc2f3a59b422bb1a9d8afa5c3ffae" candidate_uuid: "77e03eac97a3494189ee753aa206a6f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621823aae2241c3bc78c2829bc6308c" is_pre_election: true
I20260811 00:29:51.904042 19322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d7bc2f3a59b422bb1a9d8afa5c3ffae" candidate_uuid: "77e03eac97a3494189ee753aa206a6f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0beb21908b9942079c54e4a8373321a3" is_pre_election: true
I20260811 00:29:51.904407 19170 raft_consensus.cc:2454] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77e03eac97a3494189ee753aa206a6f2 in term 0.
I20260811 00:29:51.904625 19322 raft_consensus.cc:2454] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77e03eac97a3494189ee753aa206a6f2 in term 0.
I20260811 00:29:51.905357 19401 leader_election.cc:302] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0beb21908b9942079c54e4a8373321a3, 77e03eac97a3494189ee753aa206a6f2, e621823aae2241c3bc78c2829bc6308c; no voters:
I20260811 00:29:51.905907 19547 raft_consensus.cc:2781] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:29:51.906072 19547 raft_consensus.cc:482] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:29:51.906181 19547 raft_consensus.cc:3037] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:29:51.909165 19547 raft_consensus.cc:504] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } }
I20260811 00:29:51.909830 19547 leader_election.cc:288] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [CANDIDATE]: Term 1 election: Requested vote from peers 0beb21908b9942079c54e4a8373321a3 (127.2.120.2:43507), e621823aae2241c3bc78c2829bc6308c (127.2.120.1:45095)
I20260811 00:29:51.910413 19322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d7bc2f3a59b422bb1a9d8afa5c3ffae" candidate_uuid: "77e03eac97a3494189ee753aa206a6f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0beb21908b9942079c54e4a8373321a3"
I20260811 00:29:51.910478 19170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d7bc2f3a59b422bb1a9d8afa5c3ffae" candidate_uuid: "77e03eac97a3494189ee753aa206a6f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e621823aae2241c3bc78c2829bc6308c"
I20260811 00:29:51.910655 19322 raft_consensus.cc:3037] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:29:51.910768 19170 raft_consensus.cc:3037] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:29:51.913211 19170 raft_consensus.cc:2454] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77e03eac97a3494189ee753aa206a6f2 in term 1.
I20260811 00:29:51.913278 19322 raft_consensus.cc:2454] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77e03eac97a3494189ee753aa206a6f2 in term 1.
I20260811 00:29:51.913712 19401 leader_election.cc:302] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77e03eac97a3494189ee753aa206a6f2, e621823aae2241c3bc78c2829bc6308c; no voters:
I20260811 00:29:51.914076 19547 raft_consensus.cc:2781] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:29:51.914616 19547 raft_consensus.cc:686] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [term 1 LEADER]: Becoming Leader. State: Replica: 77e03eac97a3494189ee753aa206a6f2, State: Running, Role: LEADER
I20260811 00:29:51.915429 19547 consensus_queue.cc:237] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } }
W20260811 00:29:51.917917 19375 tablet.cc:2369] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:29:51.920691 19005 catalog_manager.cc:5495] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 77e03eac97a3494189ee753aa206a6f2 (127.2.120.3). New cstate: current_term: 1 leader_uuid: "77e03eac97a3494189ee753aa206a6f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77e03eac97a3494189ee753aa206a6f2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 43495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 } health_report { overall_health: UNKNOWN } } }
W20260811 00:29:51.927748 19525 tablet.cc:2369] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:29:51.961369 19537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:51.955038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42586 (local address 127.2.120.3:43495)
0811 00:29:51.955241 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:51.955247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:51.955355 (+ 108us) server_negotiation.cc:407] Connection header received
0811 00:29:51.955571 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:51.955575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:51.955660 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:51.955810 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:29:51.956294 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.957184 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.958986 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:51.959424 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:51.959536 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:51.960163 (+ 627us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:29:51.960757 (+ 594us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:29:51.960889 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:51.961068 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 00:29:51.961152 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:29:51.970654 19170 raft_consensus.cc:1261] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c [term 1 FOLLOWER]: Refusing update from remote peer 77e03eac97a3494189ee753aa206a6f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:29:51.970657 19322 raft_consensus.cc:1261] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3 [term 1 FOLLOWER]: Refusing update from remote peer 77e03eac97a3494189ee753aa206a6f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:29:51.972026 19551 consensus_queue.cc:1035] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e621823aae2241c3bc78c2829bc6308c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 45095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:29:51.972456 19547 consensus_queue.cc:1035] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0beb21908b9942079c54e4a8373321a3" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:29:51.987308 19554 mvcc.cc:205] Tried to move back new op lower bound from 7317127954298372096 to 7317127954089439232. Current Snapshot: MvccSnapshot[applied={T|T < 7317127954298372096}]
I20260811 00:29:52.004158 19007 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:33288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:29:52.004568 19007 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:33288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:29:52.008921 19007 catalog_manager.cc:5782] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69: Sending DeleteTablet for 3 replicas of tablet 8d7bc2f3a59b422bb1a9d8afa5c3ffae
I20260811 00:29:52.010254 19451 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d7bc2f3a59b422bb1a9d8afa5c3ffae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:29:52 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:42578
I20260811 00:29:52.010277 19302 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d7bc2f3a59b422bb1a9d8afa5c3ffae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:29:52 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:47404
I20260811 00:29:52.011085 19149 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d7bc2f3a59b422bb1a9d8afa5c3ffae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:29:52 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:57782
I20260811 00:29:52.011343 19566 tablet_replica.cc:330] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2: stopping tablet replica
I20260811 00:29:52.011868 19566 raft_consensus.cc:2229] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:29:52.012182 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 19053
I20260811 00:29:52.012800 19566 raft_consensus.cc:2258] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:29:52.019032 19566 ts_tablet_manager.cc:1905] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:29:52.025904 19568 tablet_replica.cc:330] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c: stopping tablet replica
I20260811 00:29:52.026451 19568 raft_consensus.cc:2229] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:29:52.027067 19568 raft_consensus.cc:2258] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:29:52.028044 19567 tablet_replica.cc:330] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3: stopping tablet replica
I20260811 00:29:52.028576 19567 raft_consensus.cc:2229] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:29:52.029168 19567 raft_consensus.cc:2258] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:29:52.031412 19568 ts_tablet_manager.cc:1905] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:29:52.034281 19538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:52.013123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57792 (local address 127.2.120.1:45095)
0811 00:29:52.013349 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:52.013359 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:52.013386 (+ 27us) server_negotiation.cc:407] Connection header received
0811 00:29:52.013648 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:52.013657 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:52.013796 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:52.013942 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:52.016632 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:52.018269 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:52.021330 (+ 3061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:52.022080 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:52.022259 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:52.026475 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:52.026491 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:52.026512 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:52.026561 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:52.030756 (+ 4195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:52.031754 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:52.031763 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:52.031766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:52.031948 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:52.032397 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:52.032402 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:52.032405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:52.032936 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:52.033255 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:52.033563 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 00:29:52.033814 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 00:29:52.035267 19215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:52.036687 19567 ts_tablet_manager.cc:1905] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:29:52.036993 19566 ts_tablet_manager.cc:1918] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:29:52.037221 19566 log.cc:1196] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/8d7bc2f3a59b422bb1a9d8afa5c3ffae
I20260811 00:29:52.037802 19566 ts_tablet_manager.cc:1939] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 77e03eac97a3494189ee753aa206a6f2: Deleting consensus metadata
I20260811 00:29:52.039636 18990 catalog_manager.cc:4841] TS 77e03eac97a3494189ee753aa206a6f2 (127.2.120.3:43495): tablet 8d7bc2f3a59b422bb1a9d8afa5c3ffae (table test-table [id=4fd6f1c6247b409d86929ece884237cb]) successfully deleted
I20260811 00:29:52.045820 19567 ts_tablet_manager.cc:1918] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:29:52.046056 19567 log.cc:1196] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/8d7bc2f3a59b422bb1a9d8afa5c3ffae
I20260811 00:29:52.046756 19567 ts_tablet_manager.cc:1939] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P 0beb21908b9942079c54e4a8373321a3: Deleting consensus metadata
I20260811 00:29:52.049019 18991 catalog_manager.cc:4841] TS 0beb21908b9942079c54e4a8373321a3 (127.2.120.2:43507): tablet 8d7bc2f3a59b422bb1a9d8afa5c3ffae (table test-table [id=4fd6f1c6247b409d86929ece884237cb]) successfully deleted
I20260811 00:29:52.169132 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 19053
I20260811 00:29:52.192826 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 19226
W20260811 00:29:52.192826 18993 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.120.1:45095 (error 108)
I20260811 00:29:52.207439 19540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:52.206901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:45095 (local address 127.0.0.1:57806)
0811 00:29:52.207105 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:52.207253 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:45095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 00:29:52.207756 18993 catalog_manager.cc:4584] TS e621823aae2241c3bc78c2829bc6308c (127.2.120.1:45095): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d7bc2f3a59b422bb1a9d8afa5c3ffae: Network error: Client connection negotiation failed: client connection to 127.2.120.1:45095: connect: Connection refused (error 111)
I20260811 00:29:52.214138 19536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:52.193755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47410 (local address 127.2.120.2:43507)
0811 00:29:52.193942 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:52.193948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:52.194144 (+ 196us) server_negotiation.cc:407] Connection header received
0811 00:29:52.194358 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:52.194368 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:52.194509 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:52.194638 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:52.197513 (+ 2875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:52.199405 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:52.201915 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:52.202897 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:52.203103 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:52.207379 (+ 4276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:52.207395 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:52.207414 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:52.207467 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:52.210709 (+ 3242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:52.211780 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:52.211788 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:52.211791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:52.211924 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:52.212426 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:52.212429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:52.212431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:52.212915 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:52.213139 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:52.213596 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 00:29:52.213773 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 00:29:52.215013 19369 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:52.261583 19540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:52.260987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:45095 (local address 127.0.0.1:57814)
0811 00:29:52.261263 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:52.261375 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:45095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260811 00:29:52.262907 19540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:52.262458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:45095 (local address 127.0.0.1:57826)
0811 00:29:52.262673 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:52.262766 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:45095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 00:29:52.308226 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 19226
I20260811 00:29:52.323603 19540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:52.323095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:45095 (local address 127.0.0.1:57832)
0811 00:29:52.323346 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:52.323439 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:45095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 00:29:52.324792 19540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:52.324403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:45095 (local address 127.0.0.1:57834)
0811 00:29:52.324590 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:52.324678 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:45095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 00:29:52.337226 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 19378
I20260811 00:29:52.356554 19537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:52.338135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42598 (local address 127.2.120.3:43495)
0811 00:29:52.338343 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 00:29:52.338348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:29:52.338449 (+ 101us) server_negotiation.cc:407] Connection header received
0811 00:29:52.338693 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:29:52.338698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:29:52.338783 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:29:52.338883 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:29:52.340987 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:52.341844 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:52.343980 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:29:52.344454 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:29:52.344688 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:29:52.348803 (+ 4115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:29:52.348824 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:29:52.348851 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:29:52.348926 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:29:52.352572 (+ 3646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:52.353794 (+ 1222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:52.353803 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:52.353806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:52.354041 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:29:52.354625 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:29:52.354630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:29:52.354632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:29:52.355044 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:29:52.355339 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 00:29:52.355824 (+ 485us) server_negotiation.cc:299] Negotiation successful
0811 00:29:52.356043 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 00:29:52.357467 19519 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:52.405786 19540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:52.405172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:45095 (local address 127.0.0.1:57838)
0811 00:29:52.405425 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:52.405535 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:45095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 00:29:52.407282 19540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:52.406778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:45095 (local address 127.0.0.1:57842)
0811 00:29:52.407049 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:52.407143 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:45095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260811 00:29:52.483363 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 19378
I20260811 00:29:52.507931 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 18971
I20260811 00:29:52.509353 19044 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:29:52.631214 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 18971
I20260811 00:29:52.660023 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:42147
--webserver_interface=127.2.120.62
--webserver_port=35241
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:42147
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:29:53.050678 19575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:53.051080 19575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:53.051260 19575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:53.058868 19575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:29:53.059007 19575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:53.059067 19575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:53.059168 19575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:29:53.059248 19575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:29:53.070971 19575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:42147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:42147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=35241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:53.072893 19575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:53.074726 19575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:53.088335 19581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:53.088248 19582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:53.088274 19584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:53.089933 19575 server_base.cc:1034] running on GCE node
I20260811 00:29:53.091014 19575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:53.093034 19575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:53.094326 19575 hybrid_clock.cc:648] HybridClock initialized: now 1786408193094284 us; error 69 us; skew 500 ppm
I20260811 00:29:53.094753 19575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:53 dist-test-slave-fd7l krb5kdc[18329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408193, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:53.098435 19575 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:29:53.100857 19575 webserver.cc:460] Webserver started at http://127.2.120.62:35241/ using document root <none> and password file <none>
I20260811 00:29:53.101722 19575 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:53.101900 19575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:53.108372 19575 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.006s
I20260811 00:29:53.115121 19592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:53.117173 19575 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 00:29:53.117406 19575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "edbb106eb3bb48949bd2ebb7b2160f69"
format_stamp: "Formatted at 2026-08-11 00:29:46 on dist-test-slave-fd7l"
server_key: "dc04bb1a3d195776772262af57992d42"
server_key_iv: "ee5d0f725197567b350d9b17232e2e8d"
server_key_version: "encryptionkey@0"
I20260811 00:29:53.118541 19575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:53.152684 19575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:53.157403 19575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:53.157760 19575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:53.178474 19575 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:42147
I20260811 00:29:53.178489 19645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:42147 every 8 connection(s)
I20260811 00:29:53.180380 19575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:29:53.188711 19646 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:29:53.188887 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 19575
I20260811 00:29:53.190016 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:45095
--local_ip_for_outbound_sockets=127.2.120.1
--tserver_master_addrs=127.2.120.62:42147
--webserver_port=33293
--webserver_interface=127.2.120.1
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:29:53.194656 19646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69: Bootstrap starting.
I20260811 00:29:53.219715 19646 log.cc:824] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69: Log is configured to *not* fsync() on all Append() calls
I20260811 00:29:53.239233 19646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:29:53.239989 19646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69: Bootstrap complete.
I20260811 00:29:53.250324 19646 raft_consensus.cc:348] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42147 } }
I20260811 00:29:53.251468 19646 raft_consensus.cc:729] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: edbb106eb3bb48949bd2ebb7b2160f69, State: Initialized, Role: FOLLOWER
I20260811 00:29:53.252177 19646 consensus_queue.cc:260] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42147 } }
I20260811 00:29:53.252482 19646 raft_consensus.cc:388] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:29:53.252671 19646 raft_consensus.cc:482] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:29:53.252888 19646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 2 FOLLOWER]: Advancing to term 3
I20260811 00:29:53.257290 19646 raft_consensus.cc:504] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42147 } }
I20260811 00:29:53.257880 19646 leader_election.cc:302] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edbb106eb3bb48949bd2ebb7b2160f69; no voters:
I20260811 00:29:53.258667 19646 leader_election.cc:288] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 00:29:53.258826 19650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 00:29:53.260085 19650 raft_consensus.cc:686] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [term 3 LEADER]: Becoming Leader. State: Replica: edbb106eb3bb48949bd2ebb7b2160f69, State: Running, Role: LEADER
I20260811 00:29:53.260825 19650 consensus_queue.cc:237] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42147 } }
I20260811 00:29:53.261292 19646 sys_catalog.cc:564] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:29:53.264413 19654 sys_catalog.cc:455] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edbb106eb3bb48949bd2ebb7b2160f69. Latest consensus state: current_term: 3 leader_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42147 } } }
I20260811 00:29:53.264158 19653 sys_catalog.cc:455] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edbb106eb3bb48949bd2ebb7b2160f69" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42147 } } }
I20260811 00:29:53.264844 19654 sys_catalog.cc:458] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [sys.catalog]: This master's current role is: LEADER
I20260811 00:29:53.264846 19653 sys_catalog.cc:458] T 00000000000000000000000000000000 P edbb106eb3bb48949bd2ebb7b2160f69 [sys.catalog]: This master's current role is: LEADER
I20260811 00:29:53.267922 19657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:29:53.278769 19657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:29:53.280740 19657 catalog_manager.cc:1259] Loaded cluster ID: 8b1cec063654483dad73038a16521570
I20260811 00:29:53.280980 19657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 00:29:53.283562 19657 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9a663619ad google::LogMessage::Fail() at ??:0
@ 0x7f9a6636488a google::LogMessage::SendToLog() at ??:0
@ 0x7f9a663614de google::LogMessage::Flush() at ??:0
@ 0x7f9a66362a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9a7bd98512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f9a7bd95976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9a7be1d610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f9a7be1d439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f9a68fad345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9a68fb091c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f9a68fb0799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f9a68f8beaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9a6c2656db start_thread at ??:0
@ 0x7f9a651fb61f clone at ??:0
W20260811 00:29:53.678685 19648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:53.679052 19648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:53.679163 19648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:53.687613 19648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:53.687749 19648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:53.687870 19648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:29:53.698549 19648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:45095
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=33293
--tserver_master_addrs=127.2.120.62:42147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:53.700578 19648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:53.702041 19648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:53.717602 19691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:53.717792 19692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:53.717602 19694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:53.719130 19648 server_base.cc:1034] running on GCE node
I20260811 00:29:53.719893 19648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:53.721199 19648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:53.722486 19648 hybrid_clock.cc:648] HybridClock initialized: now 1786408193722425 us; error 55 us; skew 500 ppm
I20260811 00:29:53.722927 19648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:53 dist-test-slave-fd7l krb5kdc[18329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408193, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:53.727020 19648 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:29:53.728962 19648 webserver.cc:460] Webserver started at http://127.2.120.1:33293/ using document root <none> and password file <none>
I20260811 00:29:53.729849 19648 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:53.730366 19648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:53.737145 19648 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:29:53.740907 19702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:53.742906 19648 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:29:53.743202 19648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "e621823aae2241c3bc78c2829bc6308c"
format_stamp: "Formatted at 2026-08-11 00:29:47 on dist-test-slave-fd7l"
server_key: "c242391d6240b69ceb669bf0e786ac12"
server_key_iv: "72050bc8694b7e25d868efc798a58a8b"
server_key_version: "encryptionkey@0"
I20260811 00:29:53.744467 19648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:53.791577 19648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:53.796268 19648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:53.796645 19648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:53.797832 19648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:29:53.799942 19712 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260811 00:29:53.806406 19712 ts_tablet_manager.cc:1852] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260811 00:29:53.807451 19712 ts_tablet_manager.cc:1905] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:29:53.815514 19712 ts_tablet_manager.cc:1918] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:29:53.815734 19712 log.cc:1196] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/8d7bc2f3a59b422bb1a9d8afa5c3ffae
I20260811 00:29:53.816274 19712 ts_tablet_manager.cc:1939] T 8d7bc2f3a59b422bb1a9d8afa5c3ffae P e621823aae2241c3bc78c2829bc6308c: Deleting consensus metadata
I20260811 00:29:53.816781 19648 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260811 00:29:53.816977 19648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.017s user 0.001s sys 0.000s
I20260811 00:29:53.817135 19648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:29:53.817235 19648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:53.856050 19648 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:45095
I20260811 00:29:53.856113 19829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:45095 every 8 connection(s)
I20260811 00:29:53.857548 19648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:29:53.859908 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 19648
I20260811 00:29:53.860695 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:43507
--local_ip_for_outbound_sockets=127.2.120.2
--tserver_master_addrs=127.2.120.62:42147
--webserver_port=38579
--webserver_interface=127.2.120.2
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:29:53.866760 19833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:53.861632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:35527)
0811 00:29:53.865384 (+ 3752us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:53.865719 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3412,"thread_start_us":3191,"threads_started":1}
W20260811 00:29:53.867902 19831 heartbeater.cc:644] Failed to heartbeat to 127.2.120.62:42147 (0 consecutive failures): Network error: Failed to ping master at 127.2.120.62:42147: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
I20260811 00:29:53.869047 19833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:53.868677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:46379)
0811 00:29:53.868844 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:53.868924 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 00:29:53.870085 19833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:53.869718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:38737)
0811 00:29:53.869864 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:53.869962 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 00:29:53.870424 19831 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:29:54.227021 19834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:54.227366 19834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:54.227545 19834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:54.235395 19834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:54.235545 19834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:54.235760 19834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:29:54.246662 19834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:43507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=38579
--tserver_master_addrs=127.2.120.62:42147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:54.248214 19834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:54.249812 19834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:54.265487 19842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:54.265588 19840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:54.265591 19839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:54.270377 19834 server_base.cc:1034] running on GCE node
I20260811 00:29:54.271256 19834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:54.272500 19834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:54.273851 19834 hybrid_clock.cc:648] HybridClock initialized: now 1786408194273808 us; error 63 us; skew 500 ppm
I20260811 00:29:54.274243 19834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:54 dist-test-slave-fd7l krb5kdc[18329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408194, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:54.282903 19834 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:29:54.284909 19834 webserver.cc:460] Webserver started at http://127.2.120.2:38579/ using document root <none> and password file <none>
I20260811 00:29:54.285800 19834 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:54.285972 19834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:54.292250 19834 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 00:29:54.295621 19849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:54.297566 19834 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:29:54.297849 19834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "0beb21908b9942079c54e4a8373321a3"
format_stamp: "Formatted at 2026-08-11 00:29:47 on dist-test-slave-fd7l"
server_key: "f0416ee26ea68cd41f825635a26f2b5a"
server_key_iv: "9e80803cb421f9903a49adbaed36f57c"
server_key_version: "encryptionkey@0"
I20260811 00:29:54.299037 19834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:54.319617 19834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:54.324054 19834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:54.324374 19834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:54.325491 19834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:29:54.326921 19834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:29:54.327050 19834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:54.327198 19834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:29:54.327287 19834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:54.364326 19834 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:43507
I20260811 00:29:54.364434 19975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:43507 every 8 connection(s)
I20260811 00:29:54.365943 19834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:29:54.371362 19978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:54.369313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:50147)
0811 00:29:54.370086 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:54.370345 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":309,"threads_started":1}
W20260811 00:29:54.372187 19976 heartbeater.cc:644] Failed to heartbeat to 127.2.120.62:42147 (0 consecutive failures): Network error: Failed to ping master at 127.2.120.62:42147: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
I20260811 00:29:54.373123 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 19834
I20260811 00:29:54.373162 19978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:54.372782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:40315)
0811 00:29:54.372947 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:54.373043 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 00:29:54.373888 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:43495
--local_ip_for_outbound_sockets=127.2.120.3
--tserver_master_addrs=127.2.120.62:42147
--webserver_port=39281
--webserver_interface=127.2.120.3
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:29:54.374190 19978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:54.373783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:41585)
0811 00:29:54.373930 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:54.374034 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 00:29:54.374418 19976 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:29:54.731813 19979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:29:54.732126 19979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:29:54.732290 19979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:29:54.739645 19979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:29:54.739784 19979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:29:54.739988 19979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:29:54.750521 19979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:43495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=39281
--tserver_master_addrs=127.2.120.62:42147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:29:54.752014 19979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:29:54.753594 19979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:29:54.768087 19985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:54.768038 19988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:29:54.768083 19986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:29:54.769311 19979 server_base.cc:1034] running on GCE node
I20260811 00:29:54.770047 19979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:29:54.771159 19979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:29:54.772430 19979 hybrid_clock.cc:648] HybridClock initialized: now 1786408194772348 us; error 66 us; skew 500 ppm
I20260811 00:29:54.772827 19979 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:29:54 dist-test-slave-fd7l krb5kdc[18329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408194, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:29:54.776711 19979 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:29:54.778307 19979 webserver.cc:460] Webserver started at http://127.2.120.3:39281/ using document root <none> and password file <none>
I20260811 00:29:54.779037 19979 fs_manager.cc:359] Metadata directory not provided
I20260811 00:29:54.779146 19979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:29:54.784844 19979 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:29:54.787858 19996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:54.789536 19979 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:29:54.789767 19979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "77e03eac97a3494189ee753aa206a6f2"
format_stamp: "Formatted at 2026-08-11 00:29:48 on dist-test-slave-fd7l"
server_key: "0adeeaaa05b7824c3eac04daaea7e0ac"
server_key_iv: "90e2254bff1634be3df41351a82396ef"
server_key_version: "encryptionkey@0"
I20260811 00:29:54.790958 19979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:29:54.809589 19979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:29:54.814523 19979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:29:54.814829 19979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:29:54.815976 19979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:29:54.817507 19979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:29:54.817627 19979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:54.817777 19979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:29:54.817847 19979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:29:54.855904 19979 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:43495
I20260811 00:29:54.855938 20126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:43495 every 8 connection(s)
I20260811 00:29:54.857904 19979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:29:54.863745 20129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:54.861569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:37755)
0811 00:29:54.862521 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:54.862762 (+ 241us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":370,"threads_started":1}
W20260811 00:29:54.864737 20127 heartbeater.cc:644] Failed to heartbeat to 127.2.120.62:42147 (0 consecutive failures): Network error: Failed to ping master at 127.2.120.62:42147: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
I20260811 00:29:54.865808 20129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:54.865390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:53925)
0811 00:29:54.865604 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:54.865682 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 00:29:54.865880 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 19979
I20260811 00:29:54.867669 20129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:54.866967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:57087)
0811 00:29:54.867282 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:54.867420 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260811 00:29:54.868260 20127 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 00:29:54.872079 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:54.871061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:43381)
0811 00:29:54.871691 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:54.871810 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":303,"threads_started":1}
I20260811 00:29:55.376344 20132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:55.375061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:50417)
0811 00:29:55.375816 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:55.376000 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":349,"threads_started":1}
I20260811 00:29:55.869606 20133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:55.868811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:51843)
0811 00:29:55.869274 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:55.869376 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":227,"threads_started":1}
I20260811 00:29:55.873869 20134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:55.873176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:52161)
0811 00:29:55.873627 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:55.873710 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":248,"threads_started":1}
I20260811 00:29:56.378499 20135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:56.377648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:39987)
0811 00:29:56.378206 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:56.378302 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":248,"threads_started":1}
I20260811 00:29:56.871296 20136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:56.870591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:36589)
0811 00:29:56.871069 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:56.871152 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":234,"threads_started":1}
I20260811 00:29:56.875545 20137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:56.874785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:56681)
0811 00:29:56.875287 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:56.875367 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":278,"threads_started":1}
I20260811 00:29:57.380160 20138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:57.379467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:44485)
0811 00:29:57.379896 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:57.379974 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":231,"threads_started":1}
I20260811 00:29:57.873116 20139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:57.872238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:45967)
0811 00:29:57.872848 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:57.872931 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":342,"threads_started":1}
I20260811 00:29:57.877393 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:57.876634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:35883)
0811 00:29:57.877144 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:57.877227 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":266,"threads_started":1}
I20260811 00:29:58.382093 20141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:58.381262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:39735)
0811 00:29:58.381798 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:58.381894 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":278,"threads_started":1}
I20260811 00:29:58.874905 20142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:58.874134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:57107)
0811 00:29:58.874638 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:58.874736 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":279,"threads_started":1}
I20260811 00:29:58.879330 20143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:58.878394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:47619)
0811 00:29:58.878986 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:58.879092 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":285,"threads_started":1}
I20260811 00:29:59.384171 20144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:59.383304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:56105)
0811 00:29:59.383867 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:59.383983 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":304,"threads_started":1}
I20260811 00:29:59.877159 20145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:59.876081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:59929)
0811 00:29:59.876777 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:59.876907 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":340,"threads_started":1}
I20260811 00:29:59.881100 20149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:29:59.880255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:37953)
0811 00:29:59.880840 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 00:29:59.880933 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":352,"threads_started":1}
I20260811 00:30:00.385938 20150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:00.385200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:47727)
0811 00:30:00.385666 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:00.385759 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":218,"threads_started":1}
I20260811 00:30:00.879504 20151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:00.878476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:51875)
0811 00:30:00.879160 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:00.879276 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":379,"threads_started":1}
I20260811 00:30:00.883045 20153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:00.882115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:51651)
0811 00:30:00.882702 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:00.882797 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":277,"threads_started":1}
I20260811 00:30:01.387892 20154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:01.387020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:47173)
0811 00:30:01.387569 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:01.387674 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":246,"threads_started":1}
I20260811 00:30:01.881763 20155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:01.880763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:43371)
0811 00:30:01.881443 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:01.881539 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":342,"threads_started":1}
I20260811 00:30:01.884959 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:01.884246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:45991)
0811 00:30:01.884721 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:01.884800 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":280,"threads_started":1}
I20260811 00:30:02.390111 20157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:02.389181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:57715)
0811 00:30:02.389825 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:02.389921 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":308,"threads_started":1}
I20260811 00:30:02.885470 20158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:02.883692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:58267)
0811 00:30:02.884794 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:02.885026 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":775,"thread_start_us":543,"threads_started":1}
I20260811 00:30:02.888993 20159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:02.886795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:54179)
0811 00:30:02.888144 (+ 1349us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:02.888447 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1061,"thread_start_us":770,"threads_started":1}
I20260811 00:30:03.392059 20160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:03.391166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:41811)
0811 00:30:03.391783 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:03.391875 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":285,"threads_started":1}
I20260811 00:30:03.887775 20161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:03.886857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:57147)
0811 00:30:03.887434 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:03.887558 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":304,"threads_started":1}
I20260811 00:30:03.892130 20162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:03.890981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:41245)
0811 00:30:03.891746 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:03.891889 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":394,"threads_started":1}
I20260811 00:30:04.393931 20163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:04.393222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:56059)
0811 00:30:04.393672 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:04.393758 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":228,"threads_started":1}
I20260811 00:30:04.889896 20164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:04.889063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:55511)
0811 00:30:04.889601 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:04.889694 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":291,"threads_started":1}
I20260811 00:30:04.894248 20165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:04.893386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:54493)
0811 00:30:04.893884 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:04.893993 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":282,"threads_started":1}
I20260811 00:30:05.395641 20167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:05.394932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:47237)
0811 00:30:05.395357 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:05.395454 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":205,"threads_started":1}
I20260811 00:30:05.891782 20168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:05.890976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:39411)
0811 00:30:05.891501 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:05.891590 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":274,"threads_started":1}
I20260811 00:30:05.896077 20169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:05.895255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:50039)
0811 00:30:05.895824 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:05.895911 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":307,"threads_started":1}
I20260811 00:30:06.397814 20170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:06.396963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:45875)
0811 00:30:06.397456 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:06.397588 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":212,"threads_started":1}
I20260811 00:30:06.893893 20171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:06.892968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:50179)
0811 00:30:06.893614 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:06.893716 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":393,"threads_started":1}
I20260811 00:30:06.898020 20172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:06.897145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:33845)
0811 00:30:06.897700 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:06.897809 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":303,"threads_started":1}
I20260811 00:30:07.399911 20173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:07.398986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:43137)
0811 00:30:07.399609 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:07.399712 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":325,"threads_started":1}
I20260811 00:30:07.896062 20174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:07.895050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:50551)
0811 00:30:07.895658 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:07.895805 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":323,"threads_started":1}
I20260811 00:30:07.899797 20175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:07.898983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:49649)
0811 00:30:07.899514 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:07.899602 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":296,"threads_started":1}
I20260811 00:30:08.401919 20176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:08.401162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:48765)
0811 00:30:08.401639 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:08.401729 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":224,"threads_started":1}
I20260811 00:30:08.897895 20177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:08.897121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:48003)
0811 00:30:08.897589 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:08.897687 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":231,"threads_started":1}
I20260811 00:30:08.902014 20178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:08.901184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:39725)
0811 00:30:08.901661 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:08.901805 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":234,"threads_started":1}
I20260811 00:30:09.404903 20179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:09.403756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.2:39859)
0811 00:30:09.404429 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:09.404625 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":343,"threads_started":1}
I20260811 00:30:09.874833 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 19648
I20260811 00:30:09.900442 20182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:09.899401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.3:54429)
0811 00:30:09.900014 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:09.900158 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":287,"threads_started":1}
I20260811 00:30:09.904099 20180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:09.876443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37692 (local address 127.2.120.1:45095)
0811 00:30:09.878629 (+ 2186us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:09.878697 (+ 68us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:09.878810 (+ 113us) server_negotiation.cc:407] Connection header received
0811 00:30:09.881224 (+ 2414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:09.881239 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:09.881822 (+ 583us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:09.882313 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:09.884006 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:09.885449 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:09.886917 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:09.887703 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:09.887932 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:09.895343 (+ 7411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:09.895355 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:09.895376 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:09.895433 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:09.899166 (+ 3733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:09.900454 (+ 1288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:09.900464 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:09.900474 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:09.900865 (+ 391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:09.901383 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:09.901387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:09.901390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:09.902140 (+ 750us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:09.902410 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:09.903020 (+ 610us) server_negotiation.cc:299] Negotiation successful
0811 00:30:09.903488 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1888,"thread_start_us":1185,"threads_started":1}
I20260811 00:30:09.904790 20184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:09.903317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:42147 (local address 127.2.120.1:52999)
0811 00:30:09.904194 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:09.904422 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":508,"threads_started":1}
I20260811 00:30:09.905925 19825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:30:09.996630 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 19648
I20260811 00:30:10.022737 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 19834
I20260811 00:30:10.042652 20186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:10.023980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39400 (local address 127.2.120.2:43507)
0811 00:30:10.024710 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:10.024725 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:10.024783 (+ 58us) server_negotiation.cc:407] Connection header received
0811 00:30:10.026374 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:10.026384 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:10.026825 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:10.027131 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:10.028873 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:10.030144 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:10.031523 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:10.032249 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:10.032404 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:10.035416 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:10.035426 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:10.035444 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:10.035490 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:10.038766 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:10.039627 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:10.039635 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:10.039643 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:10.039850 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:10.040377 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:10.040381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:10.040383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:10.041123 (+ 740us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:10.041382 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:10.041821 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 00:30:10.042335 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":383,"threads_started":1}
I20260811 00:30:10.044296 19970 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:30:10.132725 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 19834
I20260811 00:30:10.155159 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 19979
I20260811 00:30:10.177816 20189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:10.156433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59000 (local address 127.2.120.3:43495)
0811 00:30:10.157459 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:10.157475 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:10.157523 (+ 48us) server_negotiation.cc:407] Connection header received
0811 00:30:10.159227 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:10.159237 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:10.159676 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:10.159994 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:10.161836 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:10.163262 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:10.165400 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:10.166219 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:10.166391 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:10.170441 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:10.170453 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:10.170473 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:10.170533 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:10.173969 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:10.174911 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:10.174920 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:10.174928 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:10.175144 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:10.175579 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:10.175584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:10.175586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:10.176201 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:10.176495 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:10.177087 (+ 592us) server_negotiation.cc:299] Negotiation successful
0811 00:30:10.177485 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":849,"thread_start_us":664,"threads_started":1}
I20260811 00:30:10.179420 20122 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:30:10.267828 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 19979
2026-08-11T00:30:10Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25850 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:30:10 dist-test-slave-fd7l krb5kdc[20194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:30:10 dist-test-slave-fd7l krb5kdc[20194](info): set up 2 sockets
Aug 11 00:30:10 dist-test-slave-fd7l krb5kdc[20194](info): commencing operation
krb5kdc: starting...
W20260811 00:30:11.503374 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.003s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:30:11 dist-test-slave-fd7l krb5kdc[20194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:30:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:30:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:30:11.821870 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:44937
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:44937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:30:12.205302 20208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:30:12.205673 20208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:30:12.205870 20208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:30:12.217432 20208 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 00:30:12.217609 20208 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 00:30:12.217679 20208 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 00:30:12.217763 20208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:30:12.217849 20208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:30:12.217972 20208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:30:12.218073 20208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:30:12.218158 20208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:30:12.234934 20208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:44937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:44937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:30:12.236901 20208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:30:12.239344 20208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:30:12.262568 20215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:12.262791 20214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:12.265242 20217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:30:12.265425 20208 server_base.cc:1034] running on GCE node
I20260811 00:30:12.266402 20208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:30:12.268131 20208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:30:12.269407 20208 hybrid_clock.cc:648] HybridClock initialized: now 1786408212269299 us; error 91 us; skew 500 ppm
I20260811 00:30:12.269785 20208 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:30:12 dist-test-slave-fd7l krb5kdc[20194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408212, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:30:12.273439 20208 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:30:12.275902 20208 webserver.cc:460] Webserver started at http://127.2.120.62:41389/ using document root <none> and password file <none>
I20260811 00:30:12.277007 20208 fs_manager.cc:359] Metadata directory not provided
I20260811 00:30:12.277190 20208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:30:12.277585 20208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:30:12.281148 20208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "390d0abc0ac54a51b2ba81e10a16b203"
format_stamp: "Formatted at 2026-08-11 00:30:12 on dist-test-slave-fd7l"
server_key: "62658337448d8f62b69f2a4d12b06451"
server_key_iv: "c78647fb33eee83ed2958b4b028f664f"
server_key_version: "encryptionkey@0"
I20260811 00:30:12.282068 20208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "390d0abc0ac54a51b2ba81e10a16b203"
format_stamp: "Formatted at 2026-08-11 00:30:12 on dist-test-slave-fd7l"
server_key: "62658337448d8f62b69f2a4d12b06451"
server_key_iv: "c78647fb33eee83ed2958b4b028f664f"
server_key_version: "encryptionkey@0"
I20260811 00:30:12.288831 20208 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 00:30:12.294446 20224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:12.296175 20208 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:30:12.296470 20208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "390d0abc0ac54a51b2ba81e10a16b203"
format_stamp: "Formatted at 2026-08-11 00:30:12 on dist-test-slave-fd7l"
server_key: "62658337448d8f62b69f2a4d12b06451"
server_key_iv: "c78647fb33eee83ed2958b4b028f664f"
server_key_version: "encryptionkey@0"
I20260811 00:30:12.296861 20208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:30:12.317602 20208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:30:12.323630 20208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:30:12.324004 20208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:30:12.345428 20279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:44937 every 8 connection(s)
I20260811 00:30:12.345410 20208 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:44937
I20260811 00:30:12.347342 20208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:30:12.351787 20280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:30:12.356320 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 20208
I20260811 00:30:12.356635 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:30:12.357414 2528 external_mini_cluster.cc:1428] Setting key 484fa91d6ea7a5489cb50067389a4e7b
I20260811 00:30:12.364400 20280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: Bootstrap starting.
I20260811 00:30:12.368170 20280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: Neither blocks nor log segments found. Creating new log.
I20260811 00:30:12.369441 20280 log.cc:824] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: Log is configured to *not* fsync() on all Append() calls
I20260811 00:30:12.372803 20280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: No bootstrap required, opened a new log
Aug 11 00:30:12 dist-test-slave-fd7l krb5kdc[20194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:30:12.383608 20280 raft_consensus.cc:348] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "390d0abc0ac54a51b2ba81e10a16b203" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44937 } }
I20260811 00:30:12.384102 20280 raft_consensus.cc:374] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:30:12.384258 20280 raft_consensus.cc:729] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 390d0abc0ac54a51b2ba81e10a16b203, State: Initialized, Role: FOLLOWER
I20260811 00:30:12.385106 20284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:12.359851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37706 (local address 127.2.120.62:44937)
0811 00:30:12.360832 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:12.360846 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:12.360905 (+ 59us) server_negotiation.cc:407] Connection header received
0811 00:30:12.362876 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:12.362908 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:12.363400 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:12.363894 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:12.365711 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:12.367305 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:12.368967 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:12.369783 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:12.370017 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:12.374973 (+ 4956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:12.374983 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:12.375000 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:12.375064 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:12.379194 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:12.380368 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:12.380376 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:12.380388 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:12.380720 (+ 332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:12.381232 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:12.381236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:12.381238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:12.381994 (+ 756us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:12.382308 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:12.382899 (+ 591us) server_negotiation.cc:299] Negotiation successful
0811 00:30:12.383464 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":730,"thread_start_us":465,"threads_started":1}
I20260811 00:30:12.385053 20280 consensus_queue.cc:260] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "390d0abc0ac54a51b2ba81e10a16b203" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44937 } }
I20260811 00:30:12.385341 20280 raft_consensus.cc:388] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:30:12.385520 20280 raft_consensus.cc:482] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:30:12.385684 20280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:30:12.388906 20280 raft_consensus.cc:504] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "390d0abc0ac54a51b2ba81e10a16b203" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44937 } }
I20260811 00:30:12.389551 20280 leader_election.cc:302] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 390d0abc0ac54a51b2ba81e10a16b203; no voters:
I20260811 00:30:12.390331 20280 leader_election.cc:288] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:30:12.390538 20287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:30:12.392000 20287 raft_consensus.cc:686] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 1 LEADER]: Becoming Leader. State: Replica: 390d0abc0ac54a51b2ba81e10a16b203, State: Running, Role: LEADER
I20260811 00:30:12.392788 20287 consensus_queue.cc:237] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "390d0abc0ac54a51b2ba81e10a16b203" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44937 } }
I20260811 00:30:12.393288 20280 sys_catalog.cc:564] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:30:12.396204 20289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 390d0abc0ac54a51b2ba81e10a16b203. Latest consensus state: current_term: 1 leader_uuid: "390d0abc0ac54a51b2ba81e10a16b203" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "390d0abc0ac54a51b2ba81e10a16b203" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44937 } } }
I20260811 00:30:12.396341 20288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "390d0abc0ac54a51b2ba81e10a16b203" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "390d0abc0ac54a51b2ba81e10a16b203" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44937 } } }
I20260811 00:30:12.396719 20289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [sys.catalog]: This master's current role is: LEADER
I20260811 00:30:12.396809 20288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [sys.catalog]: This master's current role is: LEADER
I20260811 00:30:12.401537 20294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:30:12.408082 20294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:30:12.420811 20294 catalog_manager.cc:1347] Generated new cluster ID: 8172c854968a4832a04d266597738840
I20260811 00:30:12.420974 20294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:30:12.436566 20294 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:30:12.438118 20294 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:30:12.450212 20294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: Generated new TSK 0
I20260811 00:30:12.451335 20294 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
I20260811 00:30:12.545258 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:44937
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:30:13.021844 20310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:30:13.022264 20310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:30:13.022435 20310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:30:13.030655 20310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:30:13.030874 20310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:30:13.031085 20310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:30:13.042008 20310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:44937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:30:13.043737 20310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:30:13.045333 20310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:30:13.059592 20318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:13.059680 20316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:13.059860 20320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:30:13.060748 20310 server_base.cc:1034] running on GCE node
I20260811 00:30:13.061571 20310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:30:13.063232 20310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:30:13.064600 20310 hybrid_clock.cc:648] HybridClock initialized: now 1786408213064491 us; error 68 us; skew 500 ppm
I20260811 00:30:13.065091 20310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:30:13 dist-test-slave-fd7l krb5kdc[20194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408213, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:30:13.069200 20310 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:30:13.071076 20310 webserver.cc:460] Webserver started at http://127.2.120.1:34899/ using document root <none> and password file <none>
I20260811 00:30:13.071857 20310 fs_manager.cc:359] Metadata directory not provided
I20260811 00:30:13.072021 20310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:30:13.072367 20310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:30:13.075712 20310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "e19e85d83bf24bdc8163858886e673c2"
format_stamp: "Formatted at 2026-08-11 00:30:13 on dist-test-slave-fd7l"
server_key: "40f513607c2b6c985ecf18e24b3651a5"
server_key_iv: "432085825228936f7b28031551abc118"
server_key_version: "encryptionkey@0"
I20260811 00:30:13.076736 20310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "e19e85d83bf24bdc8163858886e673c2"
format_stamp: "Formatted at 2026-08-11 00:30:13 on dist-test-slave-fd7l"
server_key: "40f513607c2b6c985ecf18e24b3651a5"
server_key_iv: "432085825228936f7b28031551abc118"
server_key_version: "encryptionkey@0"
I20260811 00:30:13.083163 20310 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:30:13.088708 20329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:13.090366 20310 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 00:30:13.090608 20310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "e19e85d83bf24bdc8163858886e673c2"
format_stamp: "Formatted at 2026-08-11 00:30:13 on dist-test-slave-fd7l"
server_key: "40f513607c2b6c985ecf18e24b3651a5"
server_key_iv: "432085825228936f7b28031551abc118"
server_key_version: "encryptionkey@0"
I20260811 00:30:13.090864 20310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:30:13.109418 20310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:30:13.115276 20310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:30:13.115640 20310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:30:13.117086 20310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:30:13.118669 20310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:30:13.118789 20310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:13.118953 20310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:30:13.119026 20310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:13.160315 20310 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:43353
I20260811 00:30:13.160436 20453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:43353 every 8 connection(s)
I20260811 00:30:13.162040 20310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:30:13.166376 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 20310
I20260811 00:30:13.166651 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:30:13.167459 2528 external_mini_cluster.cc:1428] Setting key 6adf394a560146b274e532c8611c7b8f
Aug 11 00:30:13 dist-test-slave-fd7l krb5kdc[20194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408213, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:30:13.189299 20457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:13.166133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:49745 (local address 127.2.120.62:44937)
0811 00:30:13.166788 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:13.166795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:13.168071 (+ 1276us) server_negotiation.cc:407] Connection header received
0811 00:30:13.170542 (+ 2471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:13.170550 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:13.170662 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:13.170802 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:13.172691 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:13.174024 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:13.175997 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:13.176835 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:13.177011 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:13.181217 (+ 4206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:13.181222 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:13.181225 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:13.181265 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:13.185126 (+ 3861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:13.186381 (+ 1255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:13.186385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:13.186388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:13.186557 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:13.187179 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:13.187183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:13.187185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:13.187497 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:13.187648 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:13.188758 (+ 1110us) server_negotiation.cc:299] Negotiation successful
0811 00:30:13.188980 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":300,"threads_started":1}
I20260811 00:30:13.190977 20458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:13.166817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:49745)
0811 00:30:13.167832 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:13.167894 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 00:30:13.170183 (+ 2289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:30:13.171058 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:30:13.171068 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:30:13.171697 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:30:13.172397 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:13.172420 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:13.174294 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:13.174299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:13.175774 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:13.175794 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:13.177111 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:13.177116 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:13.177371 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:13.178183 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:30:13.178209 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:30:13.180922 (+ 2713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:30:13.185351 (+ 4429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:30:13.185365 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:30:13.185392 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:30:13.186171 (+ 779us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:30:13.186723 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:30:13.186731 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:30:13.186738 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:30:13.187032 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:30:13.187657 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:30:13.187670 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:30:13.188218 (+ 548us) client_negotiation.cc:769] Sending connection context
0811 00:30:13.188645 (+ 427us) client_negotiation.cc:240] Negotiation successful
0811 00:30:13.189318 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":420,"threads_started":1}
I20260811 00:30:13.192950 20454 heartbeater.cc:347] Connected to a master server at 127.2.120.62:44937
I20260811 00:30:13.193326 20454 heartbeater.cc:464] Registering TS with master...
I20260811 00:30:13.194299 20454 heartbeater.cc:511] Master 127.2.120.62:44937 requested a full tablet report, sending...
I20260811 00:30:13.197180 20244 ts_manager.cc:194] Registered new tserver with Master: e19e85d83bf24bdc8163858886e673c2 (127.2.120.1:43353)
I20260811 00:30:13.199515 20244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:49745
WARNING: no policy specified for kudu/127.2.120.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
I20260811 00:30:13.241474 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:44937
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:30:13.609439 20461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:30:13.609803 20461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:30:13.609973 20461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:30:13.617702 20461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:30:13.617889 20461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:30:13.618073 20461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:30:13.628346 20461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:44937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:30:13.629858 20461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:30:13.631328 20461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:30:13.647157 20467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:13.647186 20468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:13.647173 20470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:30:13.648674 20461 server_base.cc:1034] running on GCE node
I20260811 00:30:13.649382 20461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:30:13.650552 20461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:30:13.651810 20461 hybrid_clock.cc:648] HybridClock initialized: now 1786408213651767 us; error 73 us; skew 500 ppm
I20260811 00:30:13.652148 20461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:30:13 dist-test-slave-fd7l krb5kdc[20194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408213, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:30:13.656191 20461 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:30:13.658252 20461 webserver.cc:460] Webserver started at http://127.2.120.2:44237/ using document root <none> and password file <none>
I20260811 00:30:13.659093 20461 fs_manager.cc:359] Metadata directory not provided
I20260811 00:30:13.659230 20461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:30:13.659623 20461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:30:13.663378 20461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6"
format_stamp: "Formatted at 2026-08-11 00:30:13 on dist-test-slave-fd7l"
server_key: "7bdd25226c324d6ca4de3a6b4c97a8d6"
server_key_iv: "7e83bed96c171f54952eca17ac81ff52"
server_key_version: "encryptionkey@0"
I20260811 00:30:13.664543 20461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6"
format_stamp: "Formatted at 2026-08-11 00:30:13 on dist-test-slave-fd7l"
server_key: "7bdd25226c324d6ca4de3a6b4c97a8d6"
server_key_iv: "7e83bed96c171f54952eca17ac81ff52"
server_key_version: "encryptionkey@0"
I20260811 00:30:13.672020 20461 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260811 00:30:13.678483 20477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:13.680747 20461 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.001s
I20260811 00:30:13.680989 20461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6"
format_stamp: "Formatted at 2026-08-11 00:30:13 on dist-test-slave-fd7l"
server_key: "7bdd25226c324d6ca4de3a6b4c97a8d6"
server_key_iv: "7e83bed96c171f54952eca17ac81ff52"
server_key_version: "encryptionkey@0"
I20260811 00:30:13.681331 20461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:30:13.710474 20461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:30:13.715843 20461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:30:13.716228 20461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:30:13.717470 20461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:30:13.719072 20461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:30:13.719200 20461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:13.719359 20461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:30:13.719434 20461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:13.758615 20461 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:42865
I20260811 00:30:13.758682 20606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:42865 every 8 connection(s)
I20260811 00:30:13.760044 20461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:30:13.763195 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 20461
I20260811 00:30:13.763446 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:30:13.763926 2528 external_mini_cluster.cc:1428] Setting key 51f70f08461867468ef4104166bd82fc
Aug 11 00:30:13 dist-test-slave-fd7l krb5kdc[20194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408213, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:30:13.786438 20611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:13.763005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:38559 (local address 127.2.120.62:44937)
0811 00:30:13.763592 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:13.763600 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:13.764974 (+ 1374us) server_negotiation.cc:407] Connection header received
0811 00:30:13.767025 (+ 2051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:13.767034 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:13.767137 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:13.767274 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:13.769161 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:13.770529 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:13.772445 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:13.773309 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:13.773476 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:13.777805 (+ 4329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:13.777815 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:13.777822 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:13.777879 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:13.781204 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:13.782742 (+ 1538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:13.782752 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:13.782755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:13.783042 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:13.783849 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:13.783856 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:13.783867 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:13.784284 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:13.784440 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:13.785767 (+ 1327us) server_negotiation.cc:299] Negotiation successful
0811 00:30:13.786029 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":276,"threads_started":1}
I20260811 00:30:13.788160 20612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:13.763603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:38559)
0811 00:30:13.764771 (+ 1168us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:13.764841 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 00:30:13.766699 (+ 1858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:30:13.767569 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:30:13.767578 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:30:13.768167 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:30:13.768936 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:13.768958 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:13.770817 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:13.770825 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:13.772243 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:13.772260 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:13.773576 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:13.773581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:13.773839 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:13.774640 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:30:13.774667 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:30:13.777437 (+ 2770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:30:13.781447 (+ 4010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:30:13.781463 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:30:13.781491 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:30:13.782400 (+ 909us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:30:13.783238 (+ 838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:30:13.783248 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:30:13.783254 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:30:13.783598 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:30:13.784466 (+ 868us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:30:13.784480 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:30:13.785151 (+ 671us) client_negotiation.cc:769] Sending connection context
0811 00:30:13.785530 (+ 379us) client_negotiation.cc:240] Negotiation successful
0811 00:30:13.786245 (+ 715us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":339,"threads_started":1}
I20260811 00:30:13.790156 20607 heartbeater.cc:347] Connected to a master server at 127.2.120.62:44937
I20260811 00:30:13.790553 20607 heartbeater.cc:464] Registering TS with master...
I20260811 00:30:13.791424 20607 heartbeater.cc:511] Master 127.2.120.62:44937 requested a full tablet report, sending...
I20260811 00:30:13.793893 20244 ts_manager.cc:194] Registered new tserver with Master: 2b9cdf0e8b9946b1a0553af54c3c4ab6 (127.2.120.2:42865)
I20260811 00:30:13.795353 20244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:38559
WARNING: no policy specified for kudu/127.2.120.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
I20260811 00:30:13.836324 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:44937
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:30:14.203869 20454 heartbeater.cc:503] Master 127.2.120.62:44937 was elected leader, sending a full tablet report...
W20260811 00:30:14.215058 20615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:30:14.215477 20615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:30:14.215656 20615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:30:14.223482 20615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:30:14.223690 20615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:30:14.223879 20615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:30:14.234469 20615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:44937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:30:14.236018 20615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:30:14.237682 20615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:30:14.254078 20620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:14.254091 20623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:14.253949 20621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:30:14.255177 20615 server_base.cc:1034] running on GCE node
I20260811 00:30:14.256250 20615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:30:14.257805 20615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:30:14.259142 20615 hybrid_clock.cc:648] HybridClock initialized: now 1786408214259046 us; error 102 us; skew 500 ppm
I20260811 00:30:14.259828 20615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:30:14 dist-test-slave-fd7l krb5kdc[20194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408214, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:30:14.264570 20615 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:30:14.266700 20615 webserver.cc:460] Webserver started at http://127.2.120.3:35285/ using document root <none> and password file <none>
I20260811 00:30:14.267544 20615 fs_manager.cc:359] Metadata directory not provided
I20260811 00:30:14.267666 20615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:30:14.267992 20615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:30:14.271521 20615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "cd0a003570a1430baa23be0765a31c5e"
format_stamp: "Formatted at 2026-08-11 00:30:14 on dist-test-slave-fd7l"
server_key: "101ab7dedfe33b34ef2107be176a0892"
server_key_iv: "59f6777fd8f13f5f0a8c7310a7a0320f"
server_key_version: "encryptionkey@0"
I20260811 00:30:14.272871 20615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "cd0a003570a1430baa23be0765a31c5e"
format_stamp: "Formatted at 2026-08-11 00:30:14 on dist-test-slave-fd7l"
server_key: "101ab7dedfe33b34ef2107be176a0892"
server_key_iv: "59f6777fd8f13f5f0a8c7310a7a0320f"
server_key_version: "encryptionkey@0"
I20260811 00:30:14.279731 20615 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:30:14.284065 20632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:14.286437 20615 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:30:14.286636 20615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "cd0a003570a1430baa23be0765a31c5e"
format_stamp: "Formatted at 2026-08-11 00:30:14 on dist-test-slave-fd7l"
server_key: "101ab7dedfe33b34ef2107be176a0892"
server_key_iv: "59f6777fd8f13f5f0a8c7310a7a0320f"
server_key_version: "encryptionkey@0"
I20260811 00:30:14.286847 20615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:30:14.300379 20615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:30:14.305452 20615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:30:14.305799 20615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:30:14.306860 20615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:30:14.308383 20615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:30:14.308470 20615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:14.308643 20615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:30:14.308735 20615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:14.344983 20615 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:36689
I20260811 00:30:14.345069 20760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:36689 every 8 connection(s)
I20260811 00:30:14.346339 20615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:30:14.349902 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 20615
I20260811 00:30:14.350104 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:30:14.350713 2528 external_mini_cluster.cc:1428] Setting key 3a309df4f5c9111ec50b2d943d4022b8
Aug 11 00:30:14 dist-test-slave-fd7l krb5kdc[20194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408214, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:30:14.369526 20763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.349317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:41625 (local address 127.2.120.62:44937)
0811 00:30:14.350622 (+ 1305us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:14.350632 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:14.351574 (+ 942us) server_negotiation.cc:407] Connection header received
0811 00:30:14.354010 (+ 2436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:14.354015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:14.354109 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:14.354234 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:14.355828 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.356670 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.358331 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.358769 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.358916 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:14.362899 (+ 3983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:14.362907 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:14.362911 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:14.362951 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:14.366139 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:14.367109 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:14.367113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:14.367116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:14.367247 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:14.367682 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:14.367685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:14.367687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:14.367985 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:14.368111 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:14.368986 (+ 875us) server_negotiation.cc:299] Negotiation successful
0811 00:30:14.369202 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1125,"thread_start_us":1346,"threads_started":1}
I20260811 00:30:14.370504 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.349771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:41625)
0811 00:30:14.351404 (+ 1633us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:14.351465 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 00:30:14.353710 (+ 2245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:30:14.354492 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:30:14.354501 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:30:14.355042 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:30:14.355651 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:14.355670 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.356826 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.356832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:14.358162 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:14.358182 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.359019 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.359027 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:14.359309 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:14.360060 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:30:14.360085 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:30:14.362581 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:30:14.366291 (+ 3710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:30:14.366302 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:30:14.366322 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:30:14.366978 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:30:14.367360 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:30:14.367364 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:30:14.367366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:30:14.367582 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:30:14.368104 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:30:14.368112 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:30:14.368495 (+ 383us) client_negotiation.cc:769] Sending connection context
0811 00:30:14.368843 (+ 348us) client_negotiation.cc:240] Negotiation successful
0811 00:30:14.369289 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1359,"thread_start_us":1144,"threads_started":1}
I20260811 00:30:14.372392 20761 heartbeater.cc:347] Connected to a master server at 127.2.120.62:44937
I20260811 00:30:14.372833 20761 heartbeater.cc:464] Registering TS with master...
I20260811 00:30:14.374018 20761 heartbeater.cc:511] Master 127.2.120.62:44937 requested a full tablet report, sending...
I20260811 00:30:14.375998 20244 ts_manager.cc:194] Registered new tserver with Master: cd0a003570a1430baa23be0765a31c5e (127.2.120.3:36689)
I20260811 00:30:14.377029 20244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:41625
I20260811 00:30:14.380697 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:30:14.400267 20763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.386417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59640 (local address 127.2.120.62:44937)
0811 00:30:14.386718 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:14.386726 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:14.386952 (+ 226us) server_negotiation.cc:407] Connection header received
0811 00:30:14.387228 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:14.387233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:14.387301 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:14.387409 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:14.388997 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.389763 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.391182 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.391652 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.391792 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:14.394500 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:14.394507 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:14.394516 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:14.394561 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:14.397450 (+ 2889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:14.398200 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:14.398207 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:14.398210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:14.398344 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:14.398727 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:14.398732 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:14.398735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:14.399076 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:14.399330 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:14.399688 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 00:30:14.399936 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260811 00:30:14.404881 20244 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:30:14.412022 20244 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:30:14.444537 20775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.426621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39500 (local address 127.2.120.3:36689)
0811 00:30:14.427648 (+ 1027us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:14.427662 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:14.430457 (+ 2795us) server_negotiation.cc:407] Connection header received
0811 00:30:14.431541 (+ 1084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:14.431555 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:14.431945 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:14.432183 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:30:14.434083 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.435946 (+ 1863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.440824 (+ 4878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.442721 (+ 1897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.443003 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:14.443337 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:14.443803 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 00:30:14.444110 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":811,"thread_start_us":590,"threads_started":1}
I20260811 00:30:14.444590 20776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.426174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:36689 (local address 127.0.0.1:39500)
0811 00:30:14.430262 (+ 4088us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:14.430316 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 00:30:14.430935 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:30:14.432905 (+ 1970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:30:14.432915 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:30:14.432969 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:30:14.433627 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:14.433647 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.436374 (+ 2727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.436383 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:14.440491 (+ 4108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:14.440558 (+ 67us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.443153 (+ 2595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.443160 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:14.443471 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:14.443523 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 00:30:14.443712 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 00:30:14.444061 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3803,"mutex_wait_us":4,"thread_start_us":5554,"threads_started":1}
I20260811 00:30:14.446041 20777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.427459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46706 (local address 127.2.120.1:43353)
0811 00:30:14.428249 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:14.428260 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:14.432238 (+ 3978us) server_negotiation.cc:407] Connection header received
0811 00:30:14.432990 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:14.433002 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:14.433422 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:14.433699 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:30:14.437927 (+ 4228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.439618 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.442937 (+ 3319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.444602 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.444782 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:14.445058 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:14.445453 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 00:30:14.445661 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":399,"threads_started":1}
I20260811 00:30:14.446739 20774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.426728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43353 (local address 127.0.0.1:46706)
0811 00:30:14.432084 (+ 5356us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:14.432111 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 00:30:14.432393 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:30:14.436260 (+ 3867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:30:14.436274 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:30:14.436327 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:30:14.437051 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:14.437067 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.439828 (+ 2761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.439837 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:14.442711 (+ 2874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:14.442727 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.444969 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.444979 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:14.445186 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:14.445220 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 00:30:14.446122 (+ 902us) client_negotiation.cc:240] Negotiation successful
0811 00:30:14.446302 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5117,"mutex_wait_us":118,"thread_start_us":1974,"threads_started":1}
I20260811 00:30:14.449333 20685 tablet_service.cc:1467] Processing CreateTablet for tablet 15788cd25cc74c7d8d7299f1d81489ae (DEFAULT_TABLE table=test-table [id=ec0e8df7ef37406eb4e6d7e9ad039e67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:30:14.450119 20779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.429336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46090 (local address 127.2.120.2:42865)
0811 00:30:14.437903 (+ 8567us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:14.437918 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:14.438928 (+ 1010us) server_negotiation.cc:407] Connection header received
0811 00:30:14.439180 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:14.439190 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:14.439519 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:14.439764 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:30:14.441035 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.442346 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.445822 (+ 3476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.448890 (+ 3068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.449084 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:14.449313 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:14.449585 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 00:30:14.449768 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8213,"thread_start_us":8037,"threads_started":1}
I20260811 00:30:14.450117 20778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.428700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:42865 (local address 127.0.0.1:46090)
0811 00:30:14.438718 (+ 10018us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:14.438750 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 00:30:14.438971 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:30:14.440160 (+ 1189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:30:14.440167 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:30:14.440198 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:30:14.440850 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:14.440863 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.442510 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.442515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:14.444908 (+ 2393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:14.444925 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.449155 (+ 4230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.449162 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:14.449359 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:14.449399 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 00:30:14.449519 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 00:30:14.449680 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9839,"mutex_wait_us":28,"thread_start_us":9460,"threads_started":1}
I20260811 00:30:14.451218 20685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15788cd25cc74c7d8d7299f1d81489ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:30:14.451336 20380 tablet_service.cc:1467] Processing CreateTablet for tablet 15788cd25cc74c7d8d7299f1d81489ae (DEFAULT_TABLE table=test-table [id=ec0e8df7ef37406eb4e6d7e9ad039e67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:30:14.452932 20380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15788cd25cc74c7d8d7299f1d81489ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:30:14.454308 20532 tablet_service.cc:1467] Processing CreateTablet for tablet 15788cd25cc74c7d8d7299f1d81489ae (DEFAULT_TABLE table=test-table [id=ec0e8df7ef37406eb4e6d7e9ad039e67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:30:14.455708 20532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15788cd25cc74c7d8d7299f1d81489ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:30:14.478971 20781 tablet_bootstrap.cc:492] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6: Bootstrap starting.
I20260811 00:30:14.480935 20780 tablet_bootstrap.cc:492] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e: Bootstrap starting.
I20260811 00:30:14.482081 20782 tablet_bootstrap.cc:492] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2: Bootstrap starting.
I20260811 00:30:14.487735 20781 tablet_bootstrap.cc:654] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6: Neither blocks nor log segments found. Creating new log.
I20260811 00:30:14.489370 20782 tablet_bootstrap.cc:654] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2: Neither blocks nor log segments found. Creating new log.
I20260811 00:30:14.489372 20780 tablet_bootstrap.cc:654] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e: Neither blocks nor log segments found. Creating new log.
I20260811 00:30:14.490417 20781 log.cc:824] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6: Log is configured to *not* fsync() on all Append() calls
I20260811 00:30:14.491888 20780 log.cc:824] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e: Log is configured to *not* fsync() on all Append() calls
I20260811 00:30:14.491889 20782 log.cc:824] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:30:14.497239 20782 tablet_bootstrap.cc:492] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2: No bootstrap required, opened a new log
I20260811 00:30:14.497255 20781 tablet_bootstrap.cc:492] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6: No bootstrap required, opened a new log
I20260811 00:30:14.497606 20782 ts_tablet_manager.cc:1397] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2: Time spent bootstrapping tablet: real 0.016s user 0.012s sys 0.002s
I20260811 00:30:14.497639 20781 ts_tablet_manager.cc:1397] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6: Time spent bootstrapping tablet: real 0.019s user 0.016s sys 0.000s
I20260811 00:30:14.498431 20780 tablet_bootstrap.cc:492] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e: No bootstrap required, opened a new log
I20260811 00:30:14.498805 20780 ts_tablet_manager.cc:1397] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e: Time spent bootstrapping tablet: real 0.021s user 0.013s sys 0.000s
I20260811 00:30:14.506886 20782 raft_consensus.cc:348] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:30:14.507405 20782 raft_consensus.cc:374] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:30:14.507510 20782 raft_consensus.cc:729] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19e85d83bf24bdc8163858886e673c2, State: Initialized, Role: FOLLOWER
I20260811 00:30:14.508271 20782 consensus_queue.cc:260] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:30:14.510496 20782 ts_tablet_manager.cc:1428] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2: Time spent starting tablet: real 0.013s user 0.010s sys 0.003s
I20260811 00:30:14.510145 20781 raft_consensus.cc:348] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:30:14.510684 20781 raft_consensus.cc:374] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:30:14.510787 20781 raft_consensus.cc:729] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9cdf0e8b9946b1a0553af54c3c4ab6, State: Initialized, Role: FOLLOWER
I20260811 00:30:14.510895 20780 raft_consensus.cc:348] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:30:14.511370 20780 raft_consensus.cc:374] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:30:14.511482 20780 raft_consensus.cc:729] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0a003570a1430baa23be0765a31c5e, State: Initialized, Role: FOLLOWER
I20260811 00:30:14.511529 20781 consensus_queue.cc:260] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:30:14.512168 20780 consensus_queue.cc:260] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:30:14.512910 20607 heartbeater.cc:503] Master 127.2.120.62:44937 was elected leader, sending a full tablet report...
I20260811 00:30:14.514045 20781 ts_tablet_manager.cc:1428] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260811 00:30:14.515246 20761 heartbeater.cc:503] Master 127.2.120.62:44937 was elected leader, sending a full tablet report...
I20260811 00:30:14.516073 20780 ts_tablet_manager.cc:1428] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e: Time spent starting tablet: real 0.017s user 0.014s sys 0.004s
I20260811 00:30:14.544950 20788 raft_consensus.cc:482] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:30:14.545281 20788 raft_consensus.cc:504] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:30:14.547083 20788 leader_election.cc:288] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd0a003570a1430baa23be0765a31c5e (127.2.120.3:36689), 2b9cdf0e8b9946b1a0553af54c3c4ab6 (127.2.120.2:42865)
I20260811 00:30:14.556687 20779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.547713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:48687 (local address 127.2.120.2:42865)
0811 00:30:14.547969 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:14.547977 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:14.548283 (+ 306us) server_negotiation.cc:407] Connection header received
0811 00:30:14.548616 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:14.548622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:14.548742 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:14.548837 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:30:14.549535 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.550670 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.553935 (+ 3265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.555307 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.555482 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:14.555607 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:14.556187 (+ 580us) server_negotiation.cc:299] Negotiation successful
0811 00:30:14.556343 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 00:30:14.556675 20775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.548144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:53859 (local address 127.2.120.3:36689)
0811 00:30:14.548349 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:14.548355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:14.548696 (+ 341us) server_negotiation.cc:407] Connection header received
0811 00:30:14.548840 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:14.548845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:14.548918 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:14.549011 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:30:14.549718 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.550787 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.553940 (+ 3153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.555305 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.555482 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:14.555609 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:14.556187 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 00:30:14.556343 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 00:30:14.556871 20792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.547405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:42865 (local address 127.2.120.1:48687)
0811 00:30:14.548154 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:14.548183 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 00:30:14.548432 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:30:14.548883 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:30:14.548887 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:30:14.548907 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:30:14.549367 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:14.549384 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.550842 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.550873 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:14.553717 (+ 2844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:14.553736 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.555698 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.555704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:14.555957 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:14.556026 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 00:30:14.556125 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 00:30:14.556332 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"mutex_wait_us":246,"thread_start_us":369,"threads_started":1}
I20260811 00:30:14.556906 20793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.547479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:36689 (local address 127.2.120.1:53859)
0811 00:30:14.548565 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:14.548582 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:30:14.548730 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:30:14.549052 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:30:14.549058 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:30:14.549107 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:30:14.549569 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:14.549578 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.550935 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.550944 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:14.553725 (+ 2781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:14.553736 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.555677 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.555685 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:14.555958 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:14.556010 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 00:30:14.556125 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 00:30:14.556332 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"mutex_wait_us":308,"spinlock_wait_cycles":15616,"thread_start_us":707,"threads_started":1}
I20260811 00:30:14.558167 20560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15788cd25cc74c7d8d7299f1d81489ae" candidate_uuid: "e19e85d83bf24bdc8163858886e673c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" is_pre_election: true
I20260811 00:30:14.558187 20709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15788cd25cc74c7d8d7299f1d81489ae" candidate_uuid: "e19e85d83bf24bdc8163858886e673c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0a003570a1430baa23be0765a31c5e" is_pre_election: true
I20260811 00:30:14.558713 20560 raft_consensus.cc:2454] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e19e85d83bf24bdc8163858886e673c2 in term 0.
I20260811 00:30:14.558717 20709 raft_consensus.cc:2454] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e19e85d83bf24bdc8163858886e673c2 in term 0.
I20260811 00:30:14.559514 20332 leader_election.cc:302] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2b9cdf0e8b9946b1a0553af54c3c4ab6, cd0a003570a1430baa23be0765a31c5e, e19e85d83bf24bdc8163858886e673c2; no voters:
I20260811 00:30:14.560024 20788 raft_consensus.cc:2781] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:30:14.560225 20788 raft_consensus.cc:482] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:30:14.560361 20788 raft_consensus.cc:3037] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:30:14.563447 20788 raft_consensus.cc:504] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:30:14.564145 20788 leader_election.cc:288] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 1 election: Requested vote from peers cd0a003570a1430baa23be0765a31c5e (127.2.120.3:36689), 2b9cdf0e8b9946b1a0553af54c3c4ab6 (127.2.120.2:42865)
I20260811 00:30:14.564824 20560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15788cd25cc74c7d8d7299f1d81489ae" candidate_uuid: "e19e85d83bf24bdc8163858886e673c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6"
I20260811 00:30:14.564826 20709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15788cd25cc74c7d8d7299f1d81489ae" candidate_uuid: "e19e85d83bf24bdc8163858886e673c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0a003570a1430baa23be0765a31c5e"
I20260811 00:30:14.565073 20709 raft_consensus.cc:3037] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:30:14.565073 20560 raft_consensus.cc:3037] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:30:14.568625 20709 raft_consensus.cc:2454] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e19e85d83bf24bdc8163858886e673c2 in term 1.
I20260811 00:30:14.568625 20560 raft_consensus.cc:2454] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e19e85d83bf24bdc8163858886e673c2 in term 1.
I20260811 00:30:14.569288 20332 leader_election.cc:302] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd0a003570a1430baa23be0765a31c5e, e19e85d83bf24bdc8163858886e673c2; no voters:
I20260811 00:30:14.569886 20788 raft_consensus.cc:2781] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:30:14.570669 20788 raft_consensus.cc:686] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [term 1 LEADER]: Becoming Leader. State: Replica: e19e85d83bf24bdc8163858886e673c2, State: Running, Role: LEADER
I20260811 00:30:14.571282 20788 consensus_queue.cc:237] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:30:14.577124 20242 catalog_manager.cc:5495] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e19e85d83bf24bdc8163858886e673c2 (127.2.120.1). New cstate: current_term: 1 leader_uuid: "e19e85d83bf24bdc8163858886e673c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } health_report { overall_health: UNKNOWN } } }
W20260811 00:30:14.598356 20762 tablet.cc:2369] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:30:14.610172 20777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.603780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46712 (local address 127.2.120.1:43353)
0811 00:30:14.604121 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:14.604128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:14.604157 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:30:14.604396 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:14.604400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:14.604584 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:14.604765 (+ 181us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:30:14.605175 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.605948 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.607571 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.608020 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.608194 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:14.608819 (+ 625us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:30:14.609513 (+ 694us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:30:14.609680 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:14.609779 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 00:30:14.609877 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260811 00:30:14.619239 20560 raft_consensus.cc:1261] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [term 1 FOLLOWER]: Refusing update from remote peer e19e85d83bf24bdc8163858886e673c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:30:14.619239 20709 raft_consensus.cc:1261] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e [term 1 FOLLOWER]: Refusing update from remote peer e19e85d83bf24bdc8163858886e673c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:30:14.620256 20794 consensus_queue.cc:1035] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:30:14.620685 20788 consensus_queue.cc:1035] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:30:14.655824 20244 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:30:14.656162 20244 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:30:14.660347 20244 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: Sending DeleteTablet for 3 replicas of tablet 15788cd25cc74c7d8d7299f1d81489ae
I20260811 00:30:14.661820 20685 tablet_service.cc:1514] Processing DeleteTablet for tablet 15788cd25cc74c7d8d7299f1d81489ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:30:14 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:39500
I20260811 00:30:14.661823 20380 tablet_service.cc:1514] Processing DeleteTablet for tablet 15788cd25cc74c7d8d7299f1d81489ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:30:14 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:46706
I20260811 00:30:14.662734 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 20310
I20260811 00:30:14.662670 20532 tablet_service.cc:1514] Processing DeleteTablet for tablet 15788cd25cc74c7d8d7299f1d81489ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:30:14 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:46090
I20260811 00:30:14.686910 20810 tablet_replica.cc:330] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6: stopping tablet replica
I20260811 00:30:14.687680 20810 raft_consensus.cc:2229] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:30:14.688411 20810 raft_consensus.cc:2258] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:30:14.689217 20808 tablet_replica.cc:330] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2: stopping tablet replica
I20260811 00:30:14.690194 20808 raft_consensus.cc:2229] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:30:14.692942 20808 raft_consensus.cc:2258] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 11 00:30:14 dist-test-slave-fd7l krb5kdc[20194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:30:14.694391 20810 ts_tablet_manager.cc:1905] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:30:14.697355 20808 ts_tablet_manager.cc:1905] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:30:14.703086 20809 tablet_replica.cc:330] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e: stopping tablet replica
I20260811 00:30:14.703735 20809 raft_consensus.cc:2229] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:30:14.706547 20810 ts_tablet_manager.cc:1918] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:30:14.706753 20810 log.cc:1196] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/15788cd25cc74c7d8d7299f1d81489ae
I20260811 00:30:14.706840 20777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.664305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46716 (local address 127.2.120.1:43353)
0811 00:30:14.664582 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:14.664593 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:14.676095 (+ 11502us) server_negotiation.cc:407] Connection header received
0811 00:30:14.676835 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:14.676845 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:14.676975 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:14.677111 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:14.680109 (+ 2998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.681993 (+ 1884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.685203 (+ 3210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.686269 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.686455 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:14.695501 (+ 9046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:14.695525 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:14.695551 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:14.695629 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:14.702022 (+ 6393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:14.703548 (+ 1526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:14.703558 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:14.703562 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:14.703776 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:14.704409 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:14.704415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:14.704419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:14.705018 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:14.705403 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:14.706154 (+ 751us) server_negotiation.cc:299] Negotiation successful
0811 00:30:14.706452 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:30:14.707345 20810 ts_tablet_manager.cc:1939] T 15788cd25cc74c7d8d7299f1d81489ae P 2b9cdf0e8b9946b1a0553af54c3c4ab6: Deleting consensus metadata
I20260811 00:30:14.708127 20809 raft_consensus.cc:2258] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:30:14.708678 20448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:30:14.709466 20227 catalog_manager.cc:4841] TS 2b9cdf0e8b9946b1a0553af54c3c4ab6 (127.2.120.2:42865): tablet 15788cd25cc74c7d8d7299f1d81489ae (table test-table [id=ec0e8df7ef37406eb4e6d7e9ad039e67]) successfully deleted
I20260811 00:30:14.712571 20809 ts_tablet_manager.cc:1905] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:30:14.721402 20809 ts_tablet_manager.cc:1918] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:30:14.721663 20809 log.cc:1196] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/15788cd25cc74c7d8d7299f1d81489ae
I20260811 00:30:14.722288 20809 ts_tablet_manager.cc:1939] T 15788cd25cc74c7d8d7299f1d81489ae P cd0a003570a1430baa23be0765a31c5e: Deleting consensus metadata
I20260811 00:30:14.724782 20228 catalog_manager.cc:4841] TS cd0a003570a1430baa23be0765a31c5e (127.2.120.3:36689): tablet 15788cd25cc74c7d8d7299f1d81489ae (table test-table [id=ec0e8df7ef37406eb4e6d7e9ad039e67]) successfully deleted
I20260811 00:30:14.816097 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 20310
I20260811 00:30:14.840298 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 20461
W20260811 00:30:14.840984 20230 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.120.1:43353 (error 108)
I20260811 00:30:14.843650 20778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.842926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43353 (local address 127.0.0.1:46732)
0811 00:30:14.843249 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:14.843440 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 00:30:14.844148 20230 catalog_manager.cc:4584] TS e19e85d83bf24bdc8163858886e673c2 (127.2.120.1:43353): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 15788cd25cc74c7d8d7299f1d81489ae: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43353: connect: Connection refused (error 111)
Aug 11 00:30:14 dist-test-slave-fd7l krb5kdc[20194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.2@KRBTEST.COM
I20260811 00:30:14.860059 20779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.841350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46100 (local address 127.2.120.2:42865)
0811 00:30:14.842098 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:14.842105 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:14.842148 (+ 43us) server_negotiation.cc:407] Connection header received
0811 00:30:14.842229 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:14.842234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:14.842331 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:14.842480 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:14.844339 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.845387 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.846989 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.847492 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.847653 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:14.853179 (+ 5526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:14.853199 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:14.853217 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:14.853262 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:14.856827 (+ 3565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:14.857742 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:14.857750 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:14.857753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:14.857909 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:14.858364 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:14.858369 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:14.858371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:14.858842 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:14.859109 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:14.859451 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 00:30:14.859662 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596}
I20260811 00:30:14.860910 20602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:30:14.898756 20778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.897921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43353 (local address 127.0.0.1:46738)
0811 00:30:14.898285 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:14.898466 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260811 00:30:14.900593 20778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.899747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43353 (local address 127.0.0.1:46742)
0811 00:30:14.900014 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:14.900147 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 00:30:14.959132 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 20461
I20260811 00:30:14.963336 20778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.962743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43353 (local address 127.0.0.1:46748)
0811 00:30:14.963040 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:14.963160 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 00:30:14.964912 20778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.964258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43353 (local address 127.0.0.1:46754)
0811 00:30:14.964464 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:14.964700 (+ 236us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 00:30:14.984726 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 20615
Aug 11 00:30:14 dist-test-slave-fd7l krb5kdc[20194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.3@KRBTEST.COM
I20260811 00:30:15.010803 20775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:14.985614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39504 (local address 127.2.120.3:36689)
0811 00:30:14.985796 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:14.985802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:14.985825 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:30:14.985953 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:14.985958 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:14.986046 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:14.986145 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:14.987510 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.988287 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.989992 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:14.990485 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:14.990616 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:15.003988 (+ 13372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:15.004009 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:15.004028 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:15.004076 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:15.007206 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:15.008274 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:15.008289 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:15.008295 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:15.008573 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:15.009047 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:15.009051 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:15.009054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:15.009505 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:15.009743 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:15.010138 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 00:30:15.010343 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:30:15.011647 20755 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:30:15.046047 20778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:15.045372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43353 (local address 127.0.0.1:46766)
0811 00:30:15.045649 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:15.045772 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 00:30:15.047446 20778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:15.046944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43353 (local address 127.0.0.1:46774)
0811 00:30:15.047178 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:15.047289 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 00:30:15.105907 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 20615
I20260811 00:30:15.128154 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 20208
I20260811 00:30:15.129432 20275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:30:15.242550 20778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:15.241821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43353 (local address 127.0.0.1:46780)
0811 00:30:15.242072 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:15.242226 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 00:30:15.242843 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 20208
I20260811 00:30:35.272799 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:44937
--webserver_interface=127.2.120.62
--webserver_port=41389
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:44937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:30:35.661862 20817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:30:35.662271 20817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:30:35.662453 20817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:30:35.671314 20817 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 00:30:35.671520 20817 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 00:30:35.671650 20817 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 00:30:35.671749 20817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:30:35.671849 20817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:30:35.671957 20817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:30:35.672030 20817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:30:35.672111 20817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:30:35.684651 20817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:44937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:44937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=41389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:30:35.686282 20817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:30:35.687795 20817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:30:35.717710 20823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:35.717654 20827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:35.717789 20824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:30:35.723001 20817 server_base.cc:1034] running on GCE node
I20260811 00:30:35.724275 20817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:30:35.726470 20817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:30:35.727818 20817 hybrid_clock.cc:648] HybridClock initialized: now 1786408235727712 us; error 80 us; skew 500 ppm
I20260811 00:30:35.728331 20817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:30:35 dist-test-slave-fd7l krb5kdc[20194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408235, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:30:35.736591 20817 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:30:35.739822 20817 webserver.cc:460] Webserver started at http://127.2.120.62:41389/ using document root <none> and password file <none>
I20260811 00:30:35.741429 20817 fs_manager.cc:359] Metadata directory not provided
I20260811 00:30:35.741616 20817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:30:35.749398 20817 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 00:30:35.753611 20834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:35.755647 20817 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260811 00:30:35.755854 20817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "390d0abc0ac54a51b2ba81e10a16b203"
format_stamp: "Formatted at 2026-08-11 00:30:12 on dist-test-slave-fd7l"
server_key: "62658337448d8f62b69f2a4d12b06451"
server_key_iv: "c78647fb33eee83ed2958b4b028f664f"
server_key_version: "encryptionkey@0"
I20260811 00:30:35.757272 20817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:30:35.771456 20817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:30:35.776785 20817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:30:35.777200 20817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:30:35.801887 20817 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:44937
I20260811 00:30:35.801934 20894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:44937 every 8 connection(s)
I20260811 00:30:35.803869 20817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:30:35.808007 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 20817
I20260811 00:30:35.809059 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:43353
--local_ip_for_outbound_sockets=127.2.120.1
--tserver_master_addrs=127.2.120.62:44937
--webserver_port=34899
--webserver_interface=127.2.120.1
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:30:35.813892 20895 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:30:35.818167 20895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: Bootstrap starting.
I20260811 00:30:35.852500 20895 log.cc:824] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: Log is configured to *not* fsync() on all Append() calls
I20260811 00:30:35.868832 20895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:30:35.869472 20895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: Bootstrap complete.
I20260811 00:30:35.880260 20895 raft_consensus.cc:348] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "390d0abc0ac54a51b2ba81e10a16b203" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44937 } }
I20260811 00:30:35.881385 20895 raft_consensus.cc:729] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 390d0abc0ac54a51b2ba81e10a16b203, State: Initialized, Role: FOLLOWER
I20260811 00:30:35.882231 20895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "390d0abc0ac54a51b2ba81e10a16b203" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44937 } }
I20260811 00:30:35.882578 20895 raft_consensus.cc:388] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:30:35.882726 20895 raft_consensus.cc:482] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:30:35.882934 20895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 1 FOLLOWER]: Advancing to term 2
I20260811 00:30:35.887588 20895 raft_consensus.cc:504] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "390d0abc0ac54a51b2ba81e10a16b203" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44937 } }
I20260811 00:30:35.888257 20895 leader_election.cc:302] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 390d0abc0ac54a51b2ba81e10a16b203; no voters:
I20260811 00:30:35.889050 20895 leader_election.cc:288] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 00:30:35.889285 20899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 00:30:35.890722 20899 raft_consensus.cc:686] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 2 LEADER]: Becoming Leader. State: Replica: 390d0abc0ac54a51b2ba81e10a16b203, State: Running, Role: LEADER
I20260811 00:30:35.891379 20899 consensus_queue.cc:237] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "390d0abc0ac54a51b2ba81e10a16b203" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44937 } }
I20260811 00:30:35.891642 20895 sys_catalog.cc:564] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:30:35.894675 20900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "390d0abc0ac54a51b2ba81e10a16b203" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "390d0abc0ac54a51b2ba81e10a16b203" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44937 } } }
I20260811 00:30:35.894707 20901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 390d0abc0ac54a51b2ba81e10a16b203. Latest consensus state: current_term: 2 leader_uuid: "390d0abc0ac54a51b2ba81e10a16b203" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "390d0abc0ac54a51b2ba81e10a16b203" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44937 } } }
I20260811 00:30:35.895149 20900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [sys.catalog]: This master's current role is: LEADER
I20260811 00:30:35.895231 20901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [sys.catalog]: This master's current role is: LEADER
I20260811 00:30:35.905825 20907 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:30:35.918649 20907 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:30:35.921321 20907 catalog_manager.cc:1259] Loaded cluster ID: 8172c854968a4832a04d266597738840
I20260811 00:30:35.921592 20907 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:30:35.927887 20907 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:30:35.930697 20907 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: Loaded TSK: 0
I20260811 00:30:35.960465 20907 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: Generated new TSK 1
I20260811 00:30:35.961267 20907 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 00:30:36.259927 20897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:30:36.260253 20897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:30:36.260402 20897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:30:36.268424 20897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:30:36.268638 20897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:30:36.268838 20897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:30:36.279523 20897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:43353
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=34899
--tserver_master_addrs=127.2.120.62:44937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:30:36.280990 20897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:30:36.282634 20897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:30:36.299499 20926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:36.299485 20927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:36.300760 20929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:30:36.301353 20897 server_base.cc:1034] running on GCE node
I20260811 00:30:36.302271 20897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:30:36.303776 20897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:30:36.305110 20897 hybrid_clock.cc:648] HybridClock initialized: now 1786408236305032 us; error 74 us; skew 500 ppm
I20260811 00:30:36.305600 20897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:30:36 dist-test-slave-fd7l krb5kdc[20194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408236, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:30:36.310631 20897 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:30:36.312973 20897 webserver.cc:460] Webserver started at http://127.2.120.1:34899/ using document root <none> and password file <none>
I20260811 00:30:36.314134 20897 fs_manager.cc:359] Metadata directory not provided
I20260811 00:30:36.314320 20897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:30:36.323153 20897 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.003s
I20260811 00:30:36.327857 20937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:36.329744 20897 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 00:30:36.329953 20897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "e19e85d83bf24bdc8163858886e673c2"
format_stamp: "Formatted at 2026-08-11 00:30:13 on dist-test-slave-fd7l"
server_key: "40f513607c2b6c985ecf18e24b3651a5"
server_key_iv: "432085825228936f7b28031551abc118"
server_key_version: "encryptionkey@0"
I20260811 00:30:36.331305 20897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:30:36.352766 20897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:30:36.358256 20897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:30:36.358547 20897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:30:36.359694 20897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:30:36.362046 20946 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260811 00:30:36.368044 20946 ts_tablet_manager.cc:1852] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260811 00:30:36.369194 20946 ts_tablet_manager.cc:1905] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:30:36.377431 20946 ts_tablet_manager.cc:1918] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:30:36.377740 20946 log.cc:1196] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/15788cd25cc74c7d8d7299f1d81489ae
I20260811 00:30:36.378368 20946 ts_tablet_manager.cc:1939] T 15788cd25cc74c7d8d7299f1d81489ae P e19e85d83bf24bdc8163858886e673c2: Deleting consensus metadata
I20260811 00:30:36.378954 20897 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260811 00:30:36.379133 20897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.018s user 0.001s sys 0.000s
I20260811 00:30:36.379324 20897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:30:36.379427 20897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:36.418617 20897 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:43353
I20260811 00:30:36.418677 21064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:43353 every 8 connection(s)
I20260811 00:30:36.420024 20897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:30:36.421788 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 20897
I20260811 00:30:36.422533 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:42865
--local_ip_for_outbound_sockets=127.2.120.2
--tserver_master_addrs=127.2.120.62:44937
--webserver_port=44237
--webserver_interface=127.2.120.2
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:30:36 dist-test-slave-fd7l krb5kdc[20194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408236, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:30:36.447259 21067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:36.423106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:36817 (local address 127.2.120.62:44937)
0811 00:30:36.425190 (+ 2084us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:36.425208 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:36.425259 (+ 51us) server_negotiation.cc:407] Connection header received
0811 00:30:36.427233 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:36.427247 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:36.427649 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:36.428059 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:36.429795 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:36.431078 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:36.433047 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:36.433742 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:36.433991 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:36.438468 (+ 4477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:36.438497 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:36.438520 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:36.438576 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:36.441712 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:36.443032 (+ 1320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:36.443043 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:36.443049 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:36.443247 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:36.443830 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:36.443834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:36.443836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:36.444397 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:36.444652 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:36.445700 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0811 00:30:36.446280 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1843,"thread_start_us":1713,"threads_started":1}
I20260811 00:30:36.448015 21068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:36.423318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:36817)
0811 00:30:36.424223 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:36.424276 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 00:30:36.426560 (+ 2284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:30:36.428238 (+ 1678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:30:36.428251 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:30:36.428989 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:30:36.429557 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:36.429578 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:36.431288 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:36.431293 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:36.432872 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:36.432890 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:36.434039 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:36.434047 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:36.434325 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:36.435057 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:30:36.435083 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:30:36.438035 (+ 2952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:30:36.441909 (+ 3874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:30:36.441922 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:30:36.441943 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:30:36.442805 (+ 862us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:30:36.443387 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:30:36.443395 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:30:36.443401 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:30:36.443666 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:30:36.444592 (+ 926us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:30:36.444604 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:30:36.445096 (+ 492us) client_negotiation.cc:769] Sending connection context
0811 00:30:36.445452 (+ 356us) client_negotiation.cc:240] Negotiation successful
0811 00:30:36.446161 (+ 709us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":385,"threads_started":1}
I20260811 00:30:36.451767 21065 heartbeater.cc:347] Connected to a master server at 127.2.120.62:44937
I20260811 00:30:36.452239 21065 heartbeater.cc:464] Registering TS with master...
I20260811 00:30:36.453197 21065 heartbeater.cc:511] Master 127.2.120.62:44937 requested a full tablet report, sending...
I20260811 00:30:36.456140 20853 ts_manager.cc:194] Registered new tserver with Master: e19e85d83bf24bdc8163858886e673c2 (127.2.120.1:43353)
I20260811 00:30:36.458256 20853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:36817
W20260811 00:30:36.802014 21069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:30:36.802371 21069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:30:36.802557 21069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:30:36.811491 21069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:30:36.811638 21069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:30:36.811829 21069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:30:36.822731 21069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:42865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=44237
--tserver_master_addrs=127.2.120.62:44937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:30:36.824244 21069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:30:36.826130 21069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:30:36.841746 21077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:36.841717 21075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:36.841811 21074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:30:36.842846 21069 server_base.cc:1034] running on GCE node
I20260811 00:30:36.843592 21069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:30:36.844879 21069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:30:36.846168 21069 hybrid_clock.cc:648] HybridClock initialized: now 1786408236846105 us; error 55 us; skew 500 ppm
I20260811 00:30:36.846581 21069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:30:36 dist-test-slave-fd7l krb5kdc[20194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408236, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:30:36.850813 21069 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:30:36.853202 21069 webserver.cc:460] Webserver started at http://127.2.120.2:44237/ using document root <none> and password file <none>
I20260811 00:30:36.854210 21069 fs_manager.cc:359] Metadata directory not provided
I20260811 00:30:36.854415 21069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:30:36.861692 21069 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:30:36.865726 21084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:36.867501 21069 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:30:36.867755 21069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6"
format_stamp: "Formatted at 2026-08-11 00:30:13 on dist-test-slave-fd7l"
server_key: "7bdd25226c324d6ca4de3a6b4c97a8d6"
server_key_iv: "7e83bed96c171f54952eca17ac81ff52"
server_key_version: "encryptionkey@0"
I20260811 00:30:36.868954 21069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:30:36.895172 21069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:30:36.901194 21069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:30:36.901539 21069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:30:36.902791 21069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:30:36.904448 21069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:30:36.904633 21069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 00:30:36.904783 21069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:30:36.904876 21069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:36.946708 21069 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:42865
I20260811 00:30:36.946743 21213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:42865 every 8 connection(s)
I20260811 00:30:36.948689 21069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:30:36.958602 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 21069
I20260811 00:30:36.959390 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:36689
--local_ip_for_outbound_sockets=127.2.120.3
--tserver_master_addrs=127.2.120.62:44937
--webserver_port=35285
--webserver_interface=127.2.120.3
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:30:36 dist-test-slave-fd7l krb5kdc[20194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408236, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:30:36.975677 21216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:36.952001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:40179 (local address 127.2.120.62:44937)
0811 00:30:36.952634 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:36.952641 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:36.953962 (+ 1321us) server_negotiation.cc:407] Connection header received
0811 00:30:36.955954 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:36.955959 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:36.956067 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:36.956183 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:36.958519 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:36.959494 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:36.961817 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:36.962400 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:36.962831 (+ 431us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:36.967254 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:36.967290 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:36.967298 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:36.967351 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:36.971273 (+ 3922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:36.972704 (+ 1431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:36.972709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:36.972712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:36.972899 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:36.973521 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:36.973525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:36.973528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:36.973936 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:36.974101 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:36.975143 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0811 00:30:36.975366 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":295,"threads_started":1}
I20260811 00:30:36.977468 21217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:36.952584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:40179)
0811 00:30:36.953749 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:36.953794 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 00:30:36.955612 (+ 1818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:30:36.956667 (+ 1055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:30:36.956679 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:30:36.957348 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:30:36.958143 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:36.958179 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:36.959810 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:36.959816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:36.961579 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:36.961611 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:36.962943 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:36.962953 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:36.963265 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:36.964059 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:30:36.964087 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:30:36.966894 (+ 2807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:30:36.971526 (+ 4632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:30:36.971543 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:30:36.971570 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:30:36.972396 (+ 826us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:30:36.973033 (+ 637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:30:36.973041 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:30:36.973048 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:30:36.973377 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:30:36.974108 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:30:36.974120 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:30:36.974635 (+ 515us) client_negotiation.cc:769] Sending connection context
0811 00:30:36.974981 (+ 346us) client_negotiation.cc:240] Negotiation successful
0811 00:30:36.975704 (+ 723us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"thread_start_us":606,"threads_started":1}
I20260811 00:30:36.979759 21214 heartbeater.cc:347] Connected to a master server at 127.2.120.62:44937
I20260811 00:30:36.980240 21214 heartbeater.cc:464] Registering TS with master...
I20260811 00:30:36.981132 21214 heartbeater.cc:511] Master 127.2.120.62:44937 requested a full tablet report, sending...
I20260811 00:30:36.983011 20853 ts_manager.cc:194] Registered new tserver with Master: 2b9cdf0e8b9946b1a0553af54c3c4ab6 (127.2.120.2:42865)
I20260811 00:30:36.984325 20853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:40179
W20260811 00:30:37.350966 21218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:30:37.351284 21218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:30:37.351379 21218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:30:37.358642 21218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:30:37.358778 21218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:30:37.358893 21218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:30:37.369341 21218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:36689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=35285
--tserver_master_addrs=127.2.120.62:44937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:30:37.370771 21218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:30:37.372141 21218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:30:37.386615 21223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:37.386583 21224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:37.387018 21226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:30:37.387706 21218 server_base.cc:1034] running on GCE node
I20260811 00:30:37.388403 21218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:30:37.389672 21218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:30:37.391003 21218 hybrid_clock.cc:648] HybridClock initialized: now 1786408237390908 us; error 105 us; skew 500 ppm
I20260811 00:30:37.391408 21218 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:30:37 dist-test-slave-fd7l krb5kdc[20194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408237, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:30:37.395140 21218 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:30:37.396937 21218 webserver.cc:460] Webserver started at http://127.2.120.3:35285/ using document root <none> and password file <none>
I20260811 00:30:37.397753 21218 fs_manager.cc:359] Metadata directory not provided
I20260811 00:30:37.397943 21218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:30:37.404167 21218 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 00:30:37.407707 21234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:37.409399 21218 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:30:37.409646 21218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "cd0a003570a1430baa23be0765a31c5e"
format_stamp: "Formatted at 2026-08-11 00:30:14 on dist-test-slave-fd7l"
server_key: "101ab7dedfe33b34ef2107be176a0892"
server_key_iv: "59f6777fd8f13f5f0a8c7310a7a0320f"
server_key_version: "encryptionkey@0"
I20260811 00:30:37.410790 21218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:30:37.439520 21218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:30:37.444934 21218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:30:37.445278 21218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:30:37.446497 21218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:30:37.448072 21218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:30:37.448220 21218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:37.448359 21218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:30:37.448449 21218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:37.462996 21065 heartbeater.cc:503] Master 127.2.120.62:44937 was elected leader, sending a full tablet report...
I20260811 00:30:37.490883 21218 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:36689
I20260811 00:30:37.490998 21367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:36689 every 8 connection(s)
I20260811 00:30:37.492688 21218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:30:37.499190 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 21218
Aug 11 00:30:37 dist-test-slave-fd7l krb5kdc[20194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408237, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:30:37.522629 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:37.500582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51996 (local address 127.2.120.62:44937)
0811 00:30:37.501623 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:37.501627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:37.501646 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:30:37.501697 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:37.501701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:37.501757 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:37.501865 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:37.504051 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.505717 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.507965 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.508609 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.508794 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:37.513796 (+ 5002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:37.513804 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:37.513809 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:37.513857 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:37.518572 (+ 4715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:37.519572 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:37.519588 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:37.519592 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:37.519769 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:37.520309 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:37.520317 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:37.520321 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:37.520869 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:37.521188 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:37.521675 (+ 487us) server_negotiation.cc:299] Negotiation successful
0811 00:30:37.522018 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":923,"thread_start_us":709,"threads_started":1}
I20260811 00:30:37.522673 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:37.495975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:36413 (local address 127.2.120.62:44937)
0811 00:30:37.496718 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:37.496727 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:37.497968 (+ 1241us) server_negotiation.cc:407] Connection header received
0811 00:30:37.500665 (+ 2697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:37.500671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:37.500776 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:37.500904 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:37.503715 (+ 2811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.505685 (+ 1970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.508374 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.509105 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.509211 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:37.514856 (+ 5645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:37.514862 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:37.514865 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:37.514921 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:37.518791 (+ 3870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:37.519710 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:37.519721 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:37.519724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:37.519869 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:37.520352 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:37.520357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:37.520361 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:37.520841 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:37.521002 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:37.521790 (+ 788us) server_negotiation.cc:299] Negotiation successful
0811 00:30:37.522202 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":176,"server-negotiator.queue_time_us":583,"spinlock_wait_cycles":1792,"thread_start_us":356,"threads_started":1}
I20260811 00:30:37.523838 21372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:37.496899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:36413)
0811 00:30:37.497755 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:37.497798 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 00:30:37.500280 (+ 2482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:30:37.501481 (+ 1201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:30:37.501496 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:30:37.502385 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:30:37.503412 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:37.503450 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.505958 (+ 2508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.505964 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:37.508197 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:37.508216 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.509441 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.509456 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:37.509911 (+ 455us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:37.510822 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:30:37.510866 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:30:37.514553 (+ 3687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:30:37.518987 (+ 4434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:30:37.518999 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:30:37.519026 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:30:37.519570 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:30:37.520000 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:30:37.520006 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:30:37.520009 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:30:37.520228 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:30:37.520997 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:30:37.521007 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:30:37.521404 (+ 397us) client_negotiation.cc:769] Sending connection context
0811 00:30:37.521707 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 00:30:37.522317 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":350,"threads_started":1}
I20260811 00:30:37.525490 21368 heartbeater.cc:347] Connected to a master server at 127.2.120.62:44937
I20260811 00:30:37.525991 21368 heartbeater.cc:464] Registering TS with master...
I20260811 00:30:37.527055 21368 heartbeater.cc:511] Master 127.2.120.62:44937 requested a full tablet report, sending...
I20260811 00:30:37.529520 20853 ts_manager.cc:194] Registered new tserver with Master: cd0a003570a1430baa23be0765a31c5e (127.2.120.3:36689)
I20260811 00:30:37.530608 20853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:36413
I20260811 00:30:37.540244 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:30:37.549612 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:37.541489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52006 (local address 127.2.120.62:44937)
0811 00:30:37.541772 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:37.541782 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:37.542308 (+ 526us) server_negotiation.cc:407] Connection header received
0811 00:30:37.542493 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:37.542498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:37.542931 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:37.543158 (+ 227us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:30:37.543820 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.545073 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.547049 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.547724 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.547870 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:37.548623 (+ 753us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:30:37.548964 (+ 341us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 00:30:37.549309 (+ 345us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52006: token expired
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 00:30:37.549791 21374 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52006: token expired
W20260811 00:30:37.550446 20766 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.120.62:44937)
I20260811 00:30:37.566136 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:37.550992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52020 (local address 127.2.120.62:44937)
0811 00:30:37.551179 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:37.551183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:37.551221 (+ 38us) server_negotiation.cc:407] Connection header received
0811 00:30:37.551502 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:37.551507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:37.551606 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:37.551750 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:37.553494 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.554343 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.556004 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.556469 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.556708 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:37.559663 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:37.559669 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:37.559674 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:37.559719 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:37.563094 (+ 3375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:37.563988 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:37.563993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:37.563995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:37.564178 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:37.564771 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:37.564775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:37.564777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:37.565113 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:37.565331 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:37.565677 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 00:30:37.565876 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:30:37.570266 20853 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:30:37.578291 20853 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:30:37.621341 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:37.596332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41618 (local address 127.2.120.3:36689)
0811 00:30:37.611183 (+ 14851us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:37.611191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:37.611830 (+ 639us) server_negotiation.cc:407] Connection header received
0811 00:30:37.611958 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:37.611967 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:37.612272 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:37.612552 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:30:37.613694 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.615389 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.618079 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.619536 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.619676 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:37.619946 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:37.620738 (+ 792us) server_negotiation.cc:299] Negotiation successful
0811 00:30:37.620946 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14653,"thread_start_us":14485,"threads_started":1}
I20260811 00:30:37.621371 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:37.595812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40374 (local address 127.2.120.1:43353)
0811 00:30:37.611871 (+ 16059us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:37.611880 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:37.611902 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:30:37.611985 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:37.611991 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:37.612272 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:37.612539 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:30:37.613582 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.615146 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.618588 (+ 3442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.620089 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.620224 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:37.620441 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:37.620844 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 00:30:37.621022 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15893,"thread_start_us":15746,"threads_started":1}
I20260811 00:30:37.621393 21383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:37.595636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43353 (local address 127.0.0.1:40374)
0811 00:30:37.596756 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:37.596803 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 00:30:37.603370 (+ 6567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:30:37.612572 (+ 9202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:30:37.612581 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:30:37.612613 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:30:37.613271 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:37.613291 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.615276 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.615283 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:37.618350 (+ 3067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:37.618367 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.620318 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.620325 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:37.620560 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:37.620606 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 00:30:37.620673 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 00:30:37.620888 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"mutex_wait_us":172,"thread_start_us":920,"threads_started":1}
I20260811 00:30:37.621495 21384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:37.597010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:42865 (local address 127.0.0.1:48640)
0811 00:30:37.612141 (+ 15131us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:37.612159 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:30:37.612315 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:30:37.612955 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:30:37.612962 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:30:37.612989 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:30:37.613538 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:37.613552 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.615521 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.615526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:37.618352 (+ 2826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:37.618367 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.620617 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.620623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:37.620813 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:37.620846 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 00:30:37.620937 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 00:30:37.621071 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15012,"mutex_wait_us":209,"thread_start_us":14052,"threads_started":1}
I20260811 00:30:37.622377 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:37.597212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48640 (local address 127.2.120.2:42865)
0811 00:30:37.610357 (+ 13145us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:37.610373 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:37.612294 (+ 1921us) server_negotiation.cc:407] Connection header received
0811 00:30:37.612470 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:37.612480 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:37.612742 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:37.612949 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:30:37.613731 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.615359 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.618594 (+ 3235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.620079 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.620221 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:37.620406 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:37.621907 (+ 1501us) server_negotiation.cc:299] Negotiation successful
0811 00:30:37.622086 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12872,"thread_start_us":13182,"threads_started":1}
I20260811 00:30:37.622960 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:37.596260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:36689 (local address 127.0.0.1:41618)
0811 00:30:37.611341 (+ 15081us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:37.611370 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 00:30:37.611586 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:30:37.612853 (+ 1267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:30:37.612858 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:30:37.612887 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:30:37.613430 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:37.613444 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.615586 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.615590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:37.617863 (+ 2273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:37.617881 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.620105 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.620109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:37.620383 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:37.620458 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 00:30:37.620632 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 00:30:37.622699 (+ 2067us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14872,"mutex_wait_us":1050,"thread_start_us":15611,"threads_started":1}
I20260811 00:30:37.623793 20988 tablet_service.cc:1467] Processing CreateTablet for tablet 0fd983413f2c49f08ed098c476e16987 (DEFAULT_TABLE table=test-table [id=899c5eec63b8413faa3c61a5b372f704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:30:37.625440 20988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fd983413f2c49f08ed098c476e16987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:30:37.626242 21137 tablet_service.cc:1467] Processing CreateTablet for tablet 0fd983413f2c49f08ed098c476e16987 (DEFAULT_TABLE table=test-table [id=899c5eec63b8413faa3c61a5b372f704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:30:37.626750 21288 tablet_service.cc:1467] Processing CreateTablet for tablet 0fd983413f2c49f08ed098c476e16987 (DEFAULT_TABLE table=test-table [id=899c5eec63b8413faa3c61a5b372f704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:30:37.627753 21137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fd983413f2c49f08ed098c476e16987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:30:37.628065 21288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fd983413f2c49f08ed098c476e16987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:30:37.633709 21388 tablet_bootstrap.cc:492] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2: Bootstrap starting.
I20260811 00:30:37.637898 21388 tablet_bootstrap.cc:654] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2: Neither blocks nor log segments found. Creating new log.
I20260811 00:30:37.639098 21388 log.cc:824] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:30:37.642098 21388 tablet_bootstrap.cc:492] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2: No bootstrap required, opened a new log
I20260811 00:30:37.642443 21388 ts_tablet_manager.cc:1397] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 00:30:37.644537 21391 tablet_bootstrap.cc:492] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6: Bootstrap starting.
I20260811 00:30:37.646327 21390 tablet_bootstrap.cc:492] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e: Bootstrap starting.
I20260811 00:30:37.649391 21391 tablet_bootstrap.cc:654] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6: Neither blocks nor log segments found. Creating new log.
I20260811 00:30:37.650032 21390 tablet_bootstrap.cc:654] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e: Neither blocks nor log segments found. Creating new log.
I20260811 00:30:37.650594 21391 log.cc:824] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6: Log is configured to *not* fsync() on all Append() calls
I20260811 00:30:37.651244 21390 log.cc:824] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e: Log is configured to *not* fsync() on all Append() calls
I20260811 00:30:37.653278 21388 raft_consensus.cc:348] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:30:37.653807 21388 raft_consensus.cc:374] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:30:37.653932 21391 tablet_bootstrap.cc:492] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6: No bootstrap required, opened a new log
I20260811 00:30:37.653960 21388 raft_consensus.cc:729] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19e85d83bf24bdc8163858886e673c2, State: Initialized, Role: FOLLOWER
I20260811 00:30:37.654253 21391 ts_tablet_manager.cc:1397] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260811 00:30:37.654709 21388 consensus_queue.cc:260] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:30:37.655148 21390 tablet_bootstrap.cc:492] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e: No bootstrap required, opened a new log
I20260811 00:30:37.655445 21390 ts_tablet_manager.cc:1397] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 00:30:37.657008 21388 ts_tablet_manager.cc:1428] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
I20260811 00:30:37.665710 21391 raft_consensus.cc:348] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:30:37.668706 21391 raft_consensus.cc:374] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:30:37.668881 21391 raft_consensus.cc:729] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9cdf0e8b9946b1a0553af54c3c4ab6, State: Initialized, Role: FOLLOWER
I20260811 00:30:37.669417 21390 raft_consensus.cc:348] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:30:37.669719 21391 consensus_queue.cc:260] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:30:37.669991 21390 raft_consensus.cc:374] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:30:37.670128 21390 raft_consensus.cc:729] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0a003570a1430baa23be0765a31c5e, State: Initialized, Role: FOLLOWER
I20260811 00:30:37.670814 21390 consensus_queue.cc:260] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
W20260811 00:30:37.676244 21066 tablet.cc:2369] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:30:37.677174 21398 raft_consensus.cc:482] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:30:37.677479 21214 heartbeater.cc:503] Master 127.2.120.62:44937 was elected leader, sending a full tablet report...
I20260811 00:30:37.677356 21368 heartbeater.cc:503] Master 127.2.120.62:44937 was elected leader, sending a full tablet report...
I20260811 00:30:37.678822 21390 ts_tablet_manager.cc:1428] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e: Time spent starting tablet: real 0.023s user 0.015s sys 0.004s
I20260811 00:30:37.679446 21398 raft_consensus.cc:504] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:30:37.680229 21391 ts_tablet_manager.cc:1428] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6: Time spent starting tablet: real 0.026s user 0.012s sys 0.004s
I20260811 00:30:37.685292 21398 leader_election.cc:288] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e19e85d83bf24bdc8163858886e673c2 (127.2.120.1:43353), 2b9cdf0e8b9946b1a0553af54c3c4ab6 (127.2.120.2:42865)
I20260811 00:30:37.693333 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:37.685279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:57119 (local address 127.2.120.2:42865)
0811 00:30:37.685509 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:37.685515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:37.685536 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:30:37.685610 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:37.685615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:37.685692 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:37.685796 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:30:37.687576 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.688662 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.691227 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.692197 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.692363 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:37.692464 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:37.692871 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 00:30:37.693018 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 00:30:37.693344 21372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:37.682339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:42865 (local address 127.2.120.3:57119)
0811 00:30:37.682654 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:37.682677 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:30:37.682798 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:30:37.687099 (+ 4301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:30:37.687105 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:30:37.687141 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:30:37.687356 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:37.687367 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.688836 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.688842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:37.691034 (+ 2192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:37.691046 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.692398 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.692404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:37.692694 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:37.692724 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 00:30:37.692840 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 00:30:37.693030 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":5}
I20260811 00:30:37.694547 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fd983413f2c49f08ed098c476e16987" candidate_uuid: "cd0a003570a1430baa23be0765a31c5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" is_pre_election: true
I20260811 00:30:37.695122 21160 raft_consensus.cc:2454] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd0a003570a1430baa23be0765a31c5e in term 0.
I20260811 00:30:37.696075 21236 leader_election.cc:302] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b9cdf0e8b9946b1a0553af54c3c4ab6, cd0a003570a1430baa23be0765a31c5e; no voters:
I20260811 00:30:37.696238 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:37.688578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:54611 (local address 127.2.120.1:43353)
0811 00:30:37.688798 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:37.688806 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:37.689535 (+ 729us) server_negotiation.cc:407] Connection header received
0811 00:30:37.689835 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:37.689841 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:37.689923 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:37.689986 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:30:37.690898 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.691688 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.693861 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.695088 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.695208 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:37.695289 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:37.695854 (+ 565us) server_negotiation.cc:299] Negotiation successful
0811 00:30:37.696006 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 00:30:37.696405 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:37.688453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43353 (local address 127.2.120.3:54611)
0811 00:30:37.689420 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:37.689441 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:30:37.689661 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:30:37.690198 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:30:37.690203 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:30:37.690227 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:30:37.690660 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:37.690673 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.691879 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.691884 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:37.693608 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:30:37.693625 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.695325 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.695332 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:30:37.695527 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:37.695567 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 00:30:37.695681 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 00:30:37.695877 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"mutex_wait_us":3,"thread_start_us":596,"threads_started":1}
I20260811 00:30:37.696905 21399 raft_consensus.cc:2781] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:30:37.697089 21399 raft_consensus.cc:482] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:30:37.697268 21399 raft_consensus.cc:3037] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:30:37.697557 21011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fd983413f2c49f08ed098c476e16987" candidate_uuid: "cd0a003570a1430baa23be0765a31c5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19e85d83bf24bdc8163858886e673c2" is_pre_election: true
I20260811 00:30:37.698391 21011 raft_consensus.cc:2454] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd0a003570a1430baa23be0765a31c5e in term 0.
I20260811 00:30:37.700006 21399 raft_consensus.cc:504] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:30:37.700799 21399 leader_election.cc:288] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [CANDIDATE]: Term 1 election: Requested vote from peers e19e85d83bf24bdc8163858886e673c2 (127.2.120.1:43353), 2b9cdf0e8b9946b1a0553af54c3c4ab6 (127.2.120.2:42865)
I20260811 00:30:37.701390 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fd983413f2c49f08ed098c476e16987" candidate_uuid: "cd0a003570a1430baa23be0765a31c5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6"
I20260811 00:30:37.701370 21011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fd983413f2c49f08ed098c476e16987" candidate_uuid: "cd0a003570a1430baa23be0765a31c5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19e85d83bf24bdc8163858886e673c2"
W20260811 00:30:37.701450 21215 tablet.cc:2369] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:30:37.701660 21160 raft_consensus.cc:3037] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:30:37.701665 21011 raft_consensus.cc:3037] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:30:37.704129 21160 raft_consensus.cc:2454] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0a003570a1430baa23be0765a31c5e in term 1.
I20260811 00:30:37.704592 21011 raft_consensus.cc:2454] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd0a003570a1430baa23be0765a31c5e in term 1.
I20260811 00:30:37.704711 21236 leader_election.cc:302] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b9cdf0e8b9946b1a0553af54c3c4ab6, cd0a003570a1430baa23be0765a31c5e; no voters:
I20260811 00:30:37.705264 21399 raft_consensus.cc:2781] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:30:37.705469 21399 raft_consensus.cc:686] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [term 1 LEADER]: Becoming Leader. State: Replica: cd0a003570a1430baa23be0765a31c5e, State: Running, Role: LEADER
I20260811 00:30:37.706046 21399 consensus_queue.cc:237] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:30:37.711023 20853 catalog_manager.cc:5495] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e reported cstate change: term changed from 0 to 1, leader changed from <none> to cd0a003570a1430baa23be0765a31c5e (127.2.120.3). New cstate: current_term: 1 leader_uuid: "cd0a003570a1430baa23be0765a31c5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } health_report { overall_health: UNKNOWN } } }
W20260811 00:30:37.744681 21369 tablet.cc:2369] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:30:37.747267 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:37.741124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41626 (local address 127.2.120.3:36689)
0811 00:30:37.741390 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:37.741396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:37.741419 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:30:37.741664 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:37.741669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:37.741777 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:37.741971 (+ 194us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:30:37.742419 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.743235 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.745082 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.745510 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.745619 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:37.746187 (+ 568us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:30:37.746717 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:30:37.746838 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:37.746986 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 00:30:37.747068 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 00:30:37.756939 21011 raft_consensus.cc:1261] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Refusing update from remote peer cd0a003570a1430baa23be0765a31c5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:30:37.756942 21160 raft_consensus.cc:1261] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [term 1 FOLLOWER]: Refusing update from remote peer cd0a003570a1430baa23be0765a31c5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:30:37.758365 21399 consensus_queue.cc:1035] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:30:37.758749 21398 consensus_queue.cc:1035] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:30:37.773913 21404 mvcc.cc:205] Tried to move back new op lower bound from 7317128141837258752 to 7317128141648973824. Current Snapshot: MvccSnapshot[applied={T|T < 7317128141837258752}]
I20260811 00:30:37.791839 20852 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:30:37.792237 20852 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:30:37.796782 20852 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: Sending DeleteTablet for 3 replicas of tablet 0fd983413f2c49f08ed098c476e16987
I20260811 00:30:37.798318 21288 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fd983413f2c49f08ed098c476e16987 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:30:37 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:41618
I20260811 00:30:37.798313 20988 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fd983413f2c49f08ed098c476e16987 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:30:37 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:40374
I20260811 00:30:37.798395 21137 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fd983413f2c49f08ed098c476e16987 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:30:37 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:48640
I20260811 00:30:37.800971 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 20897
I20260811 00:30:37.813580 21416 tablet_replica.cc:330] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6: stopping tablet replica
I20260811 00:30:37.814445 21416 raft_consensus.cc:2229] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:30:37.815222 21416 raft_consensus.cc:2258] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:30:37.831272 21416 ts_tablet_manager.cc:1905] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:30:37.832103 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:37.802486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40382 (local address 127.2.120.1:43353)
0811 00:30:37.802658 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:37.802664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:37.802685 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:30:37.802752 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:37.802758 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:37.802855 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:37.802962 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:37.805094 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.806307 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.808585 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:37.809286 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:37.809428 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:37.815052 (+ 5624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:37.815134 (+ 82us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:37.815165 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:37.815238 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:37.821694 (+ 6456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:37.824915 (+ 3221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:37.824934 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:37.824941 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:37.825845 (+ 904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:37.826930 (+ 1085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:37.826939 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:37.826945 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:37.827731 (+ 786us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:37.828212 (+ 481us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:37.831356 (+ 3144us) server_negotiation.cc:299] Negotiation successful
0811 00:30:37.831676 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 00:30:37.832965 21060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:30:37.842417 21416 ts_tablet_manager.cc:1918] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:30:37.843618 21416 log.cc:1196] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/0fd983413f2c49f08ed098c476e16987
I20260811 00:30:37.845109 21416 ts_tablet_manager.cc:1939] T 0fd983413f2c49f08ed098c476e16987 P 2b9cdf0e8b9946b1a0553af54c3c4ab6: Deleting consensus metadata
I20260811 00:30:37.848122 20836 catalog_manager.cc:4841] TS 2b9cdf0e8b9946b1a0553af54c3c4ab6 (127.2.120.2:42865): tablet 0fd983413f2c49f08ed098c476e16987 (table test-table [id=899c5eec63b8413faa3c61a5b372f704]) successfully deleted
I20260811 00:30:37.849958 21414 tablet_replica.cc:330] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e: stopping tablet replica
I20260811 00:30:37.850585 21414 raft_consensus.cc:2229] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:30:37.851454 21414 raft_consensus.cc:2258] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:30:37.854168 21414 ts_tablet_manager.cc:1905] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:30:37.863369 21414 ts_tablet_manager.cc:1918] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:30:37.863657 21414 log.cc:1196] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/0fd983413f2c49f08ed098c476e16987
I20260811 00:30:37.864373 21414 ts_tablet_manager.cc:1939] T 0fd983413f2c49f08ed098c476e16987 P cd0a003570a1430baa23be0765a31c5e: Deleting consensus metadata
I20260811 00:30:37.866806 20839 catalog_manager.cc:4841] TS cd0a003570a1430baa23be0765a31c5e (127.2.120.3:36689): tablet 0fd983413f2c49f08ed098c476e16987 (table test-table [id=899c5eec63b8413faa3c61a5b372f704]) successfully deleted
I20260811 00:30:37.955621 21415 tablet_replica.cc:330] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2: stopping tablet replica
I20260811 00:30:37.956384 21415 raft_consensus.cc:2229] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:30:37.957086 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 20897
I20260811 00:30:37.957123 21415 raft_consensus.cc:2258] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 00:30:38.003921 20840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.120.1:43353 (error 108)
I20260811 00:30:38.004702 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 21069
I20260811 00:30:38.008453 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:38.007605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43353 (local address 127.0.0.1:40392)
0811 00:30:38.007799 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:38.007984 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 00:30:38.009032 20840 catalog_manager.cc:4584] TS e19e85d83bf24bdc8163858886e673c2 (127.2.120.1:43353): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0fd983413f2c49f08ed098c476e16987: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43353: connect: Connection refused (error 111)
I20260811 00:30:38.034168 21387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:38.007147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48642 (local address 127.2.120.2:42865)
0811 00:30:38.007460 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:38.007468 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:38.007499 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:30:38.007641 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:38.007651 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:38.007810 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:38.008003 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:38.010746 (+ 2743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:38.011767 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:38.014770 (+ 3003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:38.015632 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:38.015856 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:38.021648 (+ 5792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:38.021693 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:38.021735 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:38.021837 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:38.029698 (+ 7861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:38.031124 (+ 1426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:38.031138 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:38.031142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:38.031340 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:38.031941 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:38.031947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:38.031950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:38.032712 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:38.033074 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:38.033464 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 00:30:38.033748 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260811 00:30:38.035151 21208 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:30:38.063205 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:38.062546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43353 (local address 127.0.0.1:40398)
0811 00:30:38.062864 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:38.062989 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172}
I20260811 00:30:38.064790 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:38.064202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43353 (local address 127.0.0.1:40400)
0811 00:30:38.064481 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:38.064613 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260811 00:30:38.125648 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:38.125060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43353 (local address 127.0.0.1:40416)
0811 00:30:38.125329 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:38.125450 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 00:30:38.127429 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:38.126833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43353 (local address 127.0.0.1:40428)
0811 00:30:38.127067 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:38.127212 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 00:30:38.143092 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 21069
I20260811 00:30:38.167270 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 21218
I20260811 00:30:38.185072 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:38.168288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41642 (local address 127.2.120.3:36689)
0811 00:30:38.168499 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 00:30:38.168505 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:30:38.168719 (+ 214us) server_negotiation.cc:407] Connection header received
0811 00:30:38.168959 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:30:38.168964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:30:38.169049 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:30:38.169188 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:30:38.170656 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:38.171495 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:38.173208 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:30:38.173699 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:30:38.173875 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:30:38.176806 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:30:38.176820 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:30:38.176838 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:30:38.176891 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:30:38.180868 (+ 3977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:38.181964 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:38.181973 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:38.181976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:38.182176 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:30:38.182742 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:30:38.182746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:30:38.182749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:30:38.183403 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:30:38.183814 (+ 411us) server_negotiation.cc:1035] Waiting for connection context
0811 00:30:38.184253 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 00:30:38.184612 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 00:30:38.186211 21363 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:30:38.208990 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:38.208254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43353 (local address 127.0.0.1:40442)
0811 00:30:38.208651 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:38.208774 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
I20260811 00:30:38.210198 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:38.209769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:43353 (local address 127.0.0.1:40454)
0811 00:30:38.209999 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:38.210083 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:43353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 00:30:38.320912 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 21218
I20260811 00:30:38.346251 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 20817
I20260811 00:30:38.347635 20890 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:30:38.466360 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 20817
I20260811 00:30:58.494828 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:44937
--webserver_interface=127.2.120.62
--webserver_port=41389
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:44937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:30:58.870020 21423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:30:58.870384 21423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:30:58.870481 21423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:30:58.878137 21423 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 00:30:58.878276 21423 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 00:30:58.878317 21423 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 00:30:58.878369 21423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:30:58.878434 21423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:30:58.878507 21423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:30:58.878561 21423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:30:58.878616 21423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:30:58.889297 21423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:44937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:44937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=41389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:30:58.890707 21423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:30:58.892163 21423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:30:58.906028 21428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:58.906165 21431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:58.906167 21429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:30:58.907106 21423 server_base.cc:1034] running on GCE node
I20260811 00:30:58.907979 21423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:30:58.909715 21423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:30:58.911010 21423 hybrid_clock.cc:648] HybridClock initialized: now 1786408258910900 us; error 68 us; skew 500 ppm
I20260811 00:30:58.911443 21423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:30:58 dist-test-slave-fd7l krb5kdc[20194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408258, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:30:58.920665 21423 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:30:58.922788 21423 webserver.cc:460] Webserver started at http://127.2.120.62:41389/ using document root <none> and password file <none>
I20260811 00:30:58.923620 21423 fs_manager.cc:359] Metadata directory not provided
I20260811 00:30:58.923775 21423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:30:58.930012 21423 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:30:58.933261 21438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:58.934760 21423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:30:58.934986 21423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "390d0abc0ac54a51b2ba81e10a16b203"
format_stamp: "Formatted at 2026-08-11 00:30:12 on dist-test-slave-fd7l"
server_key: "62658337448d8f62b69f2a4d12b06451"
server_key_iv: "c78647fb33eee83ed2958b4b028f664f"
server_key_version: "encryptionkey@0"
I20260811 00:30:58.936111 21423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:30:58.962929 21423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:30:58.968047 21423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:30:58.968377 21423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:30:58.990794 21423 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:44937
I20260811 00:30:58.990835 21497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:44937 every 8 connection(s)
I20260811 00:30:58.992558 21423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:30:58.999243 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 21423
I20260811 00:30:59.000186 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:43353
--local_ip_for_outbound_sockets=127.2.120.1
--tserver_master_addrs=127.2.120.62:44937
--webserver_port=34899
--webserver_interface=127.2.120.1
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:30:59.001755 21500 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:30:59.005796 21500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: Bootstrap starting.
I20260811 00:30:59.034839 21500 log.cc:824] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: Log is configured to *not* fsync() on all Append() calls
I20260811 00:30:59.055653 21500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:30:59.056365 21500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: Bootstrap complete.
I20260811 00:30:59.067481 21500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "390d0abc0ac54a51b2ba81e10a16b203" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44937 } }
I20260811 00:30:59.068738 21500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 390d0abc0ac54a51b2ba81e10a16b203, State: Initialized, Role: FOLLOWER
I20260811 00:30:59.069680 21500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "390d0abc0ac54a51b2ba81e10a16b203" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44937 } }
I20260811 00:30:59.069984 21500 raft_consensus.cc:388] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:30:59.070120 21500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:30:59.070356 21500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 2 FOLLOWER]: Advancing to term 3
I20260811 00:30:59.075258 21500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "390d0abc0ac54a51b2ba81e10a16b203" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44937 } }
I20260811 00:30:59.075958 21500 leader_election.cc:302] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 390d0abc0ac54a51b2ba81e10a16b203; no voters:
I20260811 00:30:59.076877 21500 leader_election.cc:288] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 00:30:59.077055 21505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 00:30:59.078967 21505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [term 3 LEADER]: Becoming Leader. State: Replica: 390d0abc0ac54a51b2ba81e10a16b203, State: Running, Role: LEADER
I20260811 00:30:59.080039 21500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:30:59.079871 21505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "390d0abc0ac54a51b2ba81e10a16b203" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44937 } }
I20260811 00:30:59.083069 21506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "390d0abc0ac54a51b2ba81e10a16b203" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "390d0abc0ac54a51b2ba81e10a16b203" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44937 } } }
I20260811 00:30:59.083565 21506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [sys.catalog]: This master's current role is: LEADER
I20260811 00:30:59.084339 21507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 390d0abc0ac54a51b2ba81e10a16b203. Latest consensus state: current_term: 3 leader_uuid: "390d0abc0ac54a51b2ba81e10a16b203" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "390d0abc0ac54a51b2ba81e10a16b203" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44937 } } }
I20260811 00:30:59.084769 21507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203 [sys.catalog]: This master's current role is: LEADER
I20260811 00:30:59.090427 21513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:30:59.103410 21513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:30:59.106055 21513 catalog_manager.cc:1259] Loaded cluster ID: 8172c854968a4832a04d266597738840
I20260811 00:30:59.106383 21513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:30:59.110348 21525 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 00:30:59.112083 21513 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:30:59.113690 21513 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: Loaded TSK: 0
I20260811 00:30:59.113868 21513 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 390d0abc0ac54a51b2ba81e10a16b203: Loaded TSK: 1
F20260811 00:30:59.114916 21513 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7face20ed9ad google::LogMessage::Fail() at ??:0
@ 0x7face20f088a google::LogMessage::SendToLog() at ??:0
@ 0x7face20ed4de google::LogMessage::Flush() at ??:0
@ 0x7face20eea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7face5e517d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7face5e4073e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7facf7b201c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7facf7b1f783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7facf7bab2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7facf7bab0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7facf7bf55e9 std::function<>::operator()() at ??:0
@ 0x7facf7b23cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7facf7b21c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7facf7ba9610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7facf7ba9439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7face4d39345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7face4d3c91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7face4d3c799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7face4d17eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7face7ff16db start_thread at ??:0
@ 0x7face0f8761f clone at ??:0
W20260811 00:30:59.460330 21503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:30:59.460811 21503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:30:59.460997 21503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:30:59.473279 21503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:30:59.473461 21503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:30:59.473654 21503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:30:59.484685 21503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:43353
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=34899
--tserver_master_addrs=127.2.120.62:44937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:30:59.486191 21503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:30:59.487680 21503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:30:59.502651 21552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:59.502753 21551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:30:59.503334 21554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:30:59.503690 21503 server_base.cc:1034] running on GCE node
I20260811 00:30:59.504400 21503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:30:59.505548 21503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:30:59.506816 21503 hybrid_clock.cc:648] HybridClock initialized: now 1786408259506771 us; error 48 us; skew 500 ppm
I20260811 00:30:59.507194 21503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:30:59 dist-test-slave-fd7l krb5kdc[20194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408259, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:30:59.510896 21503 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:30:59.512719 21503 webserver.cc:460] Webserver started at http://127.2.120.1:34899/ using document root <none> and password file <none>
I20260811 00:30:59.513492 21503 fs_manager.cc:359] Metadata directory not provided
I20260811 00:30:59.513615 21503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:30:59.519748 21503 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 00:30:59.523104 21562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:30:59.524873 21503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:30:59.525053 21503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "e19e85d83bf24bdc8163858886e673c2"
format_stamp: "Formatted at 2026-08-11 00:30:13 on dist-test-slave-fd7l"
server_key: "40f513607c2b6c985ecf18e24b3651a5"
server_key_iv: "432085825228936f7b28031551abc118"
server_key_version: "encryptionkey@0"
I20260811 00:30:59.526160 21503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:30:59.557221 21503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:30:59.562290 21503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:30:59.562552 21503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:30:59.563710 21503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:30:59.565776 21570 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 00:30:59.572413 21503 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 00:30:59.572719 21503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.001s
I20260811 00:30:59.573160 21503 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 00:30:59.577584 21503 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 00:30:59.577718 21503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.005s user 0.000s sys 0.004s
I20260811 00:30:59.578202 21570 tablet_bootstrap.cc:492] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2: Bootstrap starting.
I20260811 00:30:59.608044 21570 log.cc:824] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:30:59.626355 21570 tablet_bootstrap.cc:492] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:30:59.627009 21570 tablet_bootstrap.cc:492] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2: Bootstrap complete.
I20260811 00:30:59.628600 21570 ts_tablet_manager.cc:1397] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2: Time spent bootstrapping tablet: real 0.051s user 0.021s sys 0.024s
I20260811 00:30:59.632054 21503 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:43353
I20260811 00:30:59.632215 21693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:43353 every 8 connection(s)
I20260811 00:30:59.633845 21503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:30:59.638126 21570 raft_consensus.cc:348] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:30:59.639523 21570 raft_consensus.cc:729] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19e85d83bf24bdc8163858886e673c2, State: Initialized, Role: FOLLOWER
I20260811 00:30:59.640079 21696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:59.637535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:38127)
0811 00:30:59.638615 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:59.638890 (+ 275us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":764,"thread_start_us":524,"threads_started":1}
I20260811 00:30:59.640336 21570 consensus_queue.cc:260] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
W20260811 00:30:59.641101 21694 heartbeater.cc:644] Failed to heartbeat to 127.2.120.62:44937 (0 consecutive failures): Network error: Failed to ping master at 127.2.120.62:44937: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
I20260811 00:30:59.641629 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 21503
I20260811 00:30:59.642297 21696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:59.641889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:54145)
0811 00:30:59.642042 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:59.642151 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 00:30:59.642450 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:42865
--local_ip_for_outbound_sockets=127.2.120.2
--tserver_master_addrs=127.2.120.62:44937
--webserver_port=44237
--webserver_interface=127.2.120.2
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:30:59.642917 21570 ts_tablet_manager.cc:1428] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260811 00:30:59.643601 21696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:59.643076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:44297)
0811 00:30:59.643287 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:59.643402 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 00:30:59.643875 21694 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 00:30:59.644685 21696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:30:59.644213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:43683)
0811 00:30:59.644404 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 00:30:59.644499 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 00:31:00.015007 21698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:00.015355 21698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:00.015545 21698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:00.023178 21698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:00.023362 21698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:00.023559 21698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:31:00.034291 21698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:42865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=44237
--tserver_master_addrs=127.2.120.62:44937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:00.035720 21698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:00.037174 21698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:00.053925 21708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:00.054066 21703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:00.054029 21706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:00.055281 21698 server_base.cc:1034] running on GCE node
I20260811 00:31:00.056027 21698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:00.057178 21698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:00.058393 21698 hybrid_clock.cc:648] HybridClock initialized: now 1786408260058321 us; error 72 us; skew 500 ppm
I20260811 00:31:00.058760 21698 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:00 dist-test-slave-fd7l krb5kdc[20194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408260, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:00.062417 21698 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:31:00.064081 21698 webserver.cc:460] Webserver started at http://127.2.120.2:44237/ using document root <none> and password file <none>
I20260811 00:31:00.064970 21698 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:00.065097 21698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:00.071247 21698 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260811 00:31:00.074721 21715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:00.076856 21698 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 00:31:00.077113 21698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6"
format_stamp: "Formatted at 2026-08-11 00:30:13 on dist-test-slave-fd7l"
server_key: "7bdd25226c324d6ca4de3a6b4c97a8d6"
server_key_iv: "7e83bed96c171f54952eca17ac81ff52"
server_key_version: "encryptionkey@0"
I20260811 00:31:00.078383 21698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:00.115306 21698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:00.121199 21698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:00.121587 21698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:00.122862 21698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:00.124694 21698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:00.124888 21698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:00.125049 21698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:00.125174 21698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:00.166616 21698 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:42865
I20260811 00:31:00.166764 21841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:42865 every 8 connection(s)
I20260811 00:31:00.168414 21698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:31:00.173962 21844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:00.171917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:33167)
0811 00:31:00.172808 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:00.173042 (+ 234us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":350,"threads_started":1}
W20260811 00:31:00.174747 21842 heartbeater.cc:644] Failed to heartbeat to 127.2.120.62:44937 (0 consecutive failures): Network error: Failed to ping master at 127.2.120.62:44937: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
I20260811 00:31:00.175670 21844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:00.175334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:42107)
0811 00:31:00.175485 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:00.175554 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 00:31:00.176570 21844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:00.176212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:60363)
0811 00:31:00.176336 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:00.176406 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 00:31:00.176957 21842 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 00:31:00.177237 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 21698
I20260811 00:31:00.177886 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:36689
--local_ip_for_outbound_sockets=127.2.120.3
--tserver_master_addrs=127.2.120.62:44937
--webserver_port=35285
--webserver_interface=127.2.120.3
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:31:00.562772 21845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:00.563181 21845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:00.563364 21845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:00.571314 21845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:00.571537 21845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:00.571724 21845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:31:00.582561 21845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:36689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=35285
--tserver_master_addrs=127.2.120.62:44937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:00.584247 21845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:00.585750 21845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:00.601145 21850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:00.601223 21853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:00.601176 21851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:00.602907 21845 server_base.cc:1034] running on GCE node
I20260811 00:31:00.603684 21845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:00.604866 21845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:00.606117 21845 hybrid_clock.cc:648] HybridClock initialized: now 1786408260606052 us; error 60 us; skew 500 ppm
I20260811 00:31:00.606487 21845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:00 dist-test-slave-fd7l krb5kdc[20194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408260, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:00.610544 21845 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:31:00.612308 21845 webserver.cc:460] Webserver started at http://127.2.120.3:35285/ using document root <none> and password file <none>
I20260811 00:31:00.613193 21845 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:00.613370 21845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:00.620375 21845 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:31:00.624305 21861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:00.626235 21845 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260811 00:31:00.626540 21845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "cd0a003570a1430baa23be0765a31c5e"
format_stamp: "Formatted at 2026-08-11 00:30:14 on dist-test-slave-fd7l"
server_key: "101ab7dedfe33b34ef2107be176a0892"
server_key_iv: "59f6777fd8f13f5f0a8c7310a7a0320f"
server_key_version: "encryptionkey@0"
I20260811 00:31:00.627821 21845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:00.646939 21867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:00.645788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:40309)
0811 00:31:00.646461 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:00.646609 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":367,"threads_started":1}
I20260811 00:31:00.651823 21845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:00.656879 21845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:00.657205 21845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:00.658317 21845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:00.659860 21845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:00.659993 21845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:00.660140 21845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:00.660230 21845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:00.698450 21845 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:36689
I20260811 00:31:00.698494 21985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:36689 every 8 connection(s)
I20260811 00:31:00.700217 21845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:31:00.702431 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 21845
I20260811 00:31:00.706269 21991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:00.704129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:47579)
0811 00:31:00.705051 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:00.705299 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":388,"threads_started":1}
W20260811 00:31:00.707054 21987 heartbeater.cc:644] Failed to heartbeat to 127.2.120.62:44937 (0 consecutive failures): Network error: Failed to ping master at 127.2.120.62:44937: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
I20260811 00:31:00.708019 21991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:00.707657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:46075)
0811 00:31:00.707822 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:00.707903 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 00:31:00.708984 21991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:00.708647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:58291)
0811 00:31:00.708789 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:00.708873 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 00:31:00.709241 21987 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 00:31:00.850524 21992 raft_consensus.cc:482] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:31:00.850875 21992 raft_consensus.cc:504] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:31:00.853020 21992 leader_election.cc:288] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd0a003570a1430baa23be0765a31c5e (127.2.120.3:36689), 2b9cdf0e8b9946b1a0553af54c3c4ab6 (127.2.120.2:42865)
Aug 11 00:31:00 dist-test-slave-fd7l krb5kdc[20194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408259, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.3@KRBTEST.COM
Aug 11 00:31:00 dist-test-slave-fd7l krb5kdc[20194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408259, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.2@KRBTEST.COM
I20260811 00:31:00.882736 21993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:00.853805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:45703 (local address 127.2.120.3:36689)
0811 00:31:00.855233 (+ 1428us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:00.855246 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:00.855284 (+ 38us) server_negotiation.cc:407] Connection header received
0811 00:31:00.857577 (+ 2293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:00.857591 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:00.858188 (+ 597us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:00.858630 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:00.861740 (+ 3110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:00.863770 (+ 2030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:00.866812 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:00.867996 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:00.868218 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:00.874060 (+ 5842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:00.874079 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:00.874110 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:00.874164 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:00.878279 (+ 4115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:00.879283 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:00.879293 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:00.879299 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:00.879493 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:00.879987 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:00.879992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:00.879995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:00.880689 (+ 694us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:00.880920 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:00.881766 (+ 846us) server_negotiation.cc:299] Negotiation successful
0811 00:31:00.882235 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1221,"thread_start_us":1030,"threads_started":1}
I20260811 00:31:00.882830 21995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:00.855454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:42345 (local address 127.2.120.2:42865)
0811 00:31:00.856006 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:00.856021 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:00.856060 (+ 39us) server_negotiation.cc:407] Connection header received
0811 00:31:00.858221 (+ 2161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:00.858233 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:00.858688 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:00.859034 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:00.861740 (+ 2706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:00.863770 (+ 2030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:00.866812 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:00.867996 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:00.868218 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:00.874251 (+ 6033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:00.874260 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:00.874273 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:00.874317 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:00.878146 (+ 3829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:00.879677 (+ 1531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:00.879683 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:00.879689 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:00.879876 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:00.880339 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:00.880343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:00.880346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:00.880936 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:00.881170 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:00.881764 (+ 594us) server_negotiation.cc:299] Negotiation successful
0811 00:31:00.882261 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":262,"threads_started":1}
I20260811 00:31:00.882977 21867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:00.853471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:42865 (local address 127.2.120.1:42345)
0811 00:31:00.853695 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:00.853756 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:00.854731 (+ 975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:00.859043 (+ 4312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:00.859063 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:00.860151 (+ 1088us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:00.861436 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:00.861475 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:00.864064 (+ 2589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:00.864068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:00.866589 (+ 2521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:00.866613 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:00.868327 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:00.868336 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:00.868803 (+ 467us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:00.870018 (+ 1215us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:31:00.870065 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:00.873911 (+ 3846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:00.878355 (+ 4444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:00.878371 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:00.878395 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:00.879091 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:00.880013 (+ 922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:00.880022 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:00.880028 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:00.880191 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:00.881090 (+ 899us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:00.881096 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:00.881276 (+ 180us) client_negotiation.cc:769] Sending connection context
0811 00:31:00.881473 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 00:31:00.882424 (+ 951us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":805,"spinlock_wait_cycles":28288}
I20260811 00:31:00.882861 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:00.853496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:36689 (local address 127.2.120.1:45703)
0811 00:31:00.854820 (+ 1324us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:00.854848 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:00.855019 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:00.859044 (+ 4025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:00.859057 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:00.860151 (+ 1094us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:00.861436 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:00.861475 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:00.864054 (+ 2579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:00.864061 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:00.866589 (+ 2528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:00.866613 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:00.868327 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:00.868336 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:00.868799 (+ 463us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:00.870018 (+ 1219us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:31:00.870064 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:00.873566 (+ 3502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:00.878465 (+ 4899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:00.878470 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:00.878476 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:00.879095 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:00.879623 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:00.879627 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:00.879629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:00.879834 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:00.880841 (+ 1007us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:00.880850 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:00.881188 (+ 338us) client_negotiation.cc:769] Sending connection context
0811 00:31:00.881475 (+ 287us) client_negotiation.cc:240] Negotiation successful
0811 00:31:00.882226 (+ 751us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1109,"mutex_wait_us":700,"spinlock_wait_cycles":40704,"thread_start_us":602,"threads_started":1}
W20260811 00:31:00.887296 21564 leader_election.cc:341] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2b9cdf0e8b9946b1a0553af54c3c4ab6 (127.2.120.2:42865): Not found: Tablet not found: 0fd983413f2c49f08ed098c476e16987
W20260811 00:31:00.888454 21566 leader_election.cc:341] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cd0a003570a1430baa23be0765a31c5e (127.2.120.3:36689): Not found: Tablet not found: 0fd983413f2c49f08ed098c476e16987
I20260811 00:31:00.888731 21566 leader_election.cc:302] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19e85d83bf24bdc8163858886e673c2; no voters: 2b9cdf0e8b9946b1a0553af54c3c4ab6, cd0a003570a1430baa23be0765a31c5e
I20260811 00:31:00.889317 21992 raft_consensus.cc:2726] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 00:31:01.178534 21997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:01.177579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:54193)
0811 00:31:01.178116 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:01.178274 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":262,"threads_started":1}
I20260811 00:31:01.648972 21999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:01.648078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:55071)
0811 00:31:01.648684 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:01.648776 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":256,"threads_started":1}
I20260811 00:31:01.710691 22000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:01.709881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:53269)
0811 00:31:01.710344 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:01.710471 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":237,"threads_started":1}
I20260811 00:31:02.180477 22001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:02.179681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:58377)
0811 00:31:02.180204 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:02.180296 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":293,"threads_started":1}
I20260811 00:31:02.651575 22002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:02.650353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:51331)
0811 00:31:02.651018 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:02.651142 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":359,"threads_started":1}
I20260811 00:31:02.712774 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:02.711742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:41511)
0811 00:31:02.712276 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:02.712388 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":280,"threads_started":1}
I20260811 00:31:02.763770 22004 raft_consensus.cc:482] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:31:02.764010 22004 raft_consensus.cc:504] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:31:02.765015 22004 leader_election.cc:288] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd0a003570a1430baa23be0765a31c5e (127.2.120.3:36689), 2b9cdf0e8b9946b1a0553af54c3c4ab6 (127.2.120.2:42865)
W20260811 00:31:02.766713 21564 leader_election.cc:341] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2b9cdf0e8b9946b1a0553af54c3c4ab6 (127.2.120.2:42865): Not found: Tablet not found: 0fd983413f2c49f08ed098c476e16987
W20260811 00:31:02.766957 21566 leader_election.cc:341] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cd0a003570a1430baa23be0765a31c5e (127.2.120.3:36689): Not found: Tablet not found: 0fd983413f2c49f08ed098c476e16987
I20260811 00:31:02.767081 21566 leader_election.cc:302] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19e85d83bf24bdc8163858886e673c2; no voters: 2b9cdf0e8b9946b1a0553af54c3c4ab6, cd0a003570a1430baa23be0765a31c5e
I20260811 00:31:02.767345 22004 raft_consensus.cc:2726] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 00:31:03.182580 22005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:03.181608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:56835)
0811 00:31:03.182209 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:03.182334 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":279,"threads_started":1}
I20260811 00:31:03.653949 22006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:03.653041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:36353)
0811 00:31:03.653654 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:03.653740 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":315,"threads_started":1}
I20260811 00:31:03.714782 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:03.713932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:35247)
0811 00:31:03.714462 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:03.714595 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":273,"threads_started":1}
I20260811 00:31:04.184731 22008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:04.183868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:40373)
0811 00:31:04.184386 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:04.184480 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":250,"threads_started":1}
I20260811 00:31:04.656069 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:04.655339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:59843)
0811 00:31:04.655809 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:04.655906 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":261,"threads_started":1}
I20260811 00:31:04.716849 22011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:04.715917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:46741)
0811 00:31:04.716437 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:04.716653 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":265,"threads_started":1}
I20260811 00:31:05.186689 22013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:05.185841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:51195)
0811 00:31:05.186411 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:05.186509 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":299,"threads_started":1}
I20260811 00:31:05.658772 22015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:05.657681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:34671)
0811 00:31:05.658429 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:05.658555 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":353,"threads_started":1}
I20260811 00:31:05.718883 22016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:05.718052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:53487)
0811 00:31:05.718602 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:05.718706 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":316,"threads_started":1}
I20260811 00:31:06.140270 22017 raft_consensus.cc:482] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:31:06.140520 22017 raft_consensus.cc:504] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:31:06.141350 22017 leader_election.cc:288] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd0a003570a1430baa23be0765a31c5e (127.2.120.3:36689), 2b9cdf0e8b9946b1a0553af54c3c4ab6 (127.2.120.2:42865)
W20260811 00:31:06.142488 21564 leader_election.cc:341] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2b9cdf0e8b9946b1a0553af54c3c4ab6 (127.2.120.2:42865): Not found: Tablet not found: 0fd983413f2c49f08ed098c476e16987
W20260811 00:31:06.142738 21566 leader_election.cc:341] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cd0a003570a1430baa23be0765a31c5e (127.2.120.3:36689): Not found: Tablet not found: 0fd983413f2c49f08ed098c476e16987
I20260811 00:31:06.142881 21566 leader_election.cc:302] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19e85d83bf24bdc8163858886e673c2; no voters: 2b9cdf0e8b9946b1a0553af54c3c4ab6, cd0a003570a1430baa23be0765a31c5e
I20260811 00:31:06.143203 22017 raft_consensus.cc:2726] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 00:31:06.188782 22018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:06.187813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:45003)
0811 00:31:06.188384 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:06.188485 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":313,"threads_started":1}
I20260811 00:31:06.660938 22019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:06.659976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:57331)
0811 00:31:06.660637 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:06.660742 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":299,"threads_started":1}
I20260811 00:31:06.720757 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:06.719930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:44373)
0811 00:31:06.720451 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:06.720561 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":270,"threads_started":1}
I20260811 00:31:07.190626 22021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:07.189774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:42175)
0811 00:31:07.190328 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:07.190412 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":253,"threads_started":1}
I20260811 00:31:07.663097 22023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:07.662067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:54791)
0811 00:31:07.662785 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:07.662870 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":482,"threads_started":1}
I20260811 00:31:07.722872 22025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:07.721883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:50923)
0811 00:31:07.722515 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:07.722643 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":311,"threads_started":1}
I20260811 00:31:08.192361 22026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:08.191622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:42555)
0811 00:31:08.192090 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:08.192170 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":261,"threads_started":1}
I20260811 00:31:08.665099 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:08.664246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:36951)
0811 00:31:08.664832 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:08.664919 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":358,"threads_started":1}
I20260811 00:31:08.724758 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:08.723929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:45073)
0811 00:31:08.724397 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:08.724487 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":249,"threads_started":1}
I20260811 00:31:09.194878 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:09.193732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:33885)
0811 00:31:09.194422 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:09.194626 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":293,"threads_started":1}
I20260811 00:31:09.667114 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:09.666282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:34571)
0811 00:31:09.666813 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:09.666913 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":293,"threads_started":1}
I20260811 00:31:09.726729 22031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:09.725848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:55227)
0811 00:31:09.726450 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:09.726558 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":313,"threads_started":1}
I20260811 00:31:10.197283 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:10.196340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:54671)
0811 00:31:10.196963 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:10.197064 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":380,"threads_started":1}
I20260811 00:31:10.563400 22033 raft_consensus.cc:482] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:31:10.563678 22033 raft_consensus.cc:504] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:31:10.564443 22033 leader_election.cc:288] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd0a003570a1430baa23be0765a31c5e (127.2.120.3:36689), 2b9cdf0e8b9946b1a0553af54c3c4ab6 (127.2.120.2:42865)
W20260811 00:31:10.565609 21564 leader_election.cc:341] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2b9cdf0e8b9946b1a0553af54c3c4ab6 (127.2.120.2:42865): Not found: Tablet not found: 0fd983413f2c49f08ed098c476e16987
W20260811 00:31:10.565822 21566 leader_election.cc:341] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cd0a003570a1430baa23be0765a31c5e (127.2.120.3:36689): Not found: Tablet not found: 0fd983413f2c49f08ed098c476e16987
I20260811 00:31:10.565987 21566 leader_election.cc:302] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19e85d83bf24bdc8163858886e673c2; no voters: 2b9cdf0e8b9946b1a0553af54c3c4ab6, cd0a003570a1430baa23be0765a31c5e
I20260811 00:31:10.566344 22033 raft_consensus.cc:2726] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 00:31:10.669814 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:10.668751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:44561)
0811 00:31:10.669424 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:10.669576 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":345,"threads_started":1}
I20260811 00:31:10.728812 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:10.727920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:59929)
0811 00:31:10.728439 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:10.728620 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":278,"threads_started":1}
I20260811 00:31:11.199504 22036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:11.198576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:43965)
0811 00:31:11.199188 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:11.199310 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":310,"threads_started":1}
I20260811 00:31:11.672493 22037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:11.671526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:44925)
0811 00:31:11.672139 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:11.672245 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":296,"threads_started":1}
I20260811 00:31:11.730923 22038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:11.730016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:40339)
0811 00:31:11.730621 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:11.730733 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":293,"threads_started":1}
I20260811 00:31:12.201939 22039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:12.200824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:49927)
0811 00:31:12.201511 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:12.201693 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":357,"threads_started":1}
I20260811 00:31:12.674722 22040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:12.673848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:58637)
0811 00:31:12.674454 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:12.674554 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":335,"threads_started":1}
I20260811 00:31:12.733047 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:12.732084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:56265)
0811 00:31:12.732635 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:12.732778 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":245,"threads_started":1}
I20260811 00:31:13.204663 22042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:13.203337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:47181)
0811 00:31:13.204270 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:13.204376 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":815,"thread_start_us":490,"threads_started":1}
I20260811 00:31:13.677398 22043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:13.676257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:44533)
0811 00:31:13.676989 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:13.677085 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":448,"threads_started":1}
I20260811 00:31:13.735060 22044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:13.734161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:45921)
0811 00:31:13.734786 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:13.734885 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":360,"threads_started":1}
I20260811 00:31:13.768267 22045 raft_consensus.cc:482] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:31:13.768477 22045 raft_consensus.cc:504] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e85d83bf24bdc8163858886e673c2" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 43353 } } peers { permanent_uuid: "cd0a003570a1430baa23be0765a31c5e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 36689 } } peers { permanent_uuid: "2b9cdf0e8b9946b1a0553af54c3c4ab6" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42865 } }
I20260811 00:31:13.769284 22045 leader_election.cc:288] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd0a003570a1430baa23be0765a31c5e (127.2.120.3:36689), 2b9cdf0e8b9946b1a0553af54c3c4ab6 (127.2.120.2:42865)
W20260811 00:31:13.770310 21566 leader_election.cc:341] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cd0a003570a1430baa23be0765a31c5e (127.2.120.3:36689): Not found: Tablet not found: 0fd983413f2c49f08ed098c476e16987
W20260811 00:31:13.770524 21564 leader_election.cc:341] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2b9cdf0e8b9946b1a0553af54c3c4ab6 (127.2.120.2:42865): Not found: Tablet not found: 0fd983413f2c49f08ed098c476e16987
I20260811 00:31:13.770632 21564 leader_election.cc:302] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19e85d83bf24bdc8163858886e673c2; no voters: 2b9cdf0e8b9946b1a0553af54c3c4ab6, cd0a003570a1430baa23be0765a31c5e
I20260811 00:31:13.770877 22045 raft_consensus.cc:2726] T 0fd983413f2c49f08ed098c476e16987 P e19e85d83bf24bdc8163858886e673c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 00:31:14.206696 22046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:14.205869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:49867)
0811 00:31:14.206391 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:14.206481 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":292,"threads_started":1}
I20260811 00:31:14.679702 22047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:14.678827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:46867)
0811 00:31:14.679391 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:14.679503 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":290,"threads_started":1}
I20260811 00:31:14.737215 22049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:14.736217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:53845)
0811 00:31:14.736867 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:14.736964 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":367,"threads_started":1}
I20260811 00:31:15.209165 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:15.208115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.2:34363)
0811 00:31:15.208838 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:15.208973 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":274,"threads_started":1}
I20260811 00:31:15.681623 22051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:15.680817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.1:60801)
0811 00:31:15.681319 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:15.681406 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":261,"threads_started":1}
I20260811 00:31:15.711863 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 21503
I20260811 00:31:15.735285 22052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:15.713304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32908 (local address 127.2.120.1:43353)
0811 00:31:15.714063 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:15.714077 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:15.714346 (+ 269us) server_negotiation.cc:407] Connection header received
0811 00:31:15.714576 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:15.714587 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:15.714851 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:15.715042 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:15.716584 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:15.717681 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:15.719336 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:15.719974 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:15.720177 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:15.727758 (+ 7581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:15.727780 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:15.727801 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:15.727853 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:15.731729 (+ 3876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:15.732848 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:15.732859 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:15.732863 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:15.733101 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:15.733514 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:15.733518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:15.733520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:15.733969 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:15.734263 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:15.734598 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 00:31:15.734844 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":308,"threads_started":1}
I20260811 00:31:15.736917 21689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:15.742655 22056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:15.739156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44937 (local address 127.2.120.3:54857)
0811 00:31:15.742280 (+ 3124us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:15.742436 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.62:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2916,"thread_start_us":2723,"threads_started":1}
I20260811 00:31:15.844691 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 21503
I20260811 00:31:15.872818 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 21698
I20260811 00:31:15.889904 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:15.873761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56272 (local address 127.2.120.2:42865)
0811 00:31:15.874275 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:15.874280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:15.874298 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:31:15.874358 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:15.874363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:15.874427 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:15.874527 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:15.876152 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:15.877162 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:15.879024 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:15.879824 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:15.879954 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:15.883260 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:15.883269 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:15.883273 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:15.883326 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:15.887035 (+ 3709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:15.887857 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:15.887862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:15.887864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:15.888015 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:15.888388 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:15.888392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:15.888393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:15.888813 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:15.889033 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:15.889388 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 00:31:15.889574 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":316,"threads_started":1}
I20260811 00:31:15.890653 21837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:15.985076 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 21698
I20260811 00:31:16.014448 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 21845
I20260811 00:31:16.035701 22059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:16.015406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59436 (local address 127.2.120.3:36689)
0811 00:31:16.016034 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:16.016039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:16.016057 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:31:16.016122 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:16.016126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:16.016204 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:16.016341 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:16.018204 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:16.019113 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:16.020605 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:16.021195 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:16.021392 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:16.025673 (+ 4281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:16.025682 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:16.025688 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:16.025765 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:16.030188 (+ 4423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:16.031638 (+ 1450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:16.031647 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:16.031650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:16.031923 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:16.032838 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:16.032846 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:16.032848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:16.033476 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:16.034174 (+ 698us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:16.034710 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 00:31:16.035067 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":342,"threads_started":1}
I20260811 00:31:16.036865 21981 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:16.137532 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 21845
2026-08-11T00:31:16Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65863 ms)
[----------] 30 tests from SecurityITest (336832 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:31:16 dist-test-slave-fd7l krb5kdc[22063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:31:16 dist-test-slave-fd7l krb5kdc[22063](info): set up 2 sockets
Aug 11 00:31:16 dist-test-slave-fd7l krb5kdc[22063](info): commencing operation
krb5kdc: starting...
W20260811 00:31:17.401024 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:31:17 dist-test-slave-fd7l krb5kdc[22063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:31:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:31:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:31:17.753491 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:45819
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:36021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:45819
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:31:18.165803 22076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:18.166189 22076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:18.166355 22076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:18.175791 22076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:31:18.175968 22076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:18.176023 22076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:18.176177 22076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:31:18.176241 22076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:31:18.190212 22076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:36021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:45819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:45819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:18.191900 22076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:18.193516 22076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:18.209017 22087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:18.209174 22083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:18.209146 22085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:18.210825 22076 server_base.cc:1034] running on GCE node
I20260811 00:31:18.211947 22076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:18.214040 22076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:18.215354 22076 hybrid_clock.cc:648] HybridClock initialized: now 1786408278215295 us; error 45 us; skew 500 ppm
I20260811 00:31:18.215900 22076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:18 dist-test-slave-fd7l krb5kdc[22063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408278, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:18.220192 22076 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:31:18.222353 22076 webserver.cc:460] Webserver started at http://127.2.120.62:34759/ using document root <none> and password file <none>
I20260811 00:31:18.223265 22076 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:18.223466 22076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:18.223860 22076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:18.227705 22076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "e952bc995e4044c289a305da41f7034e"
format_stamp: "Formatted at 2026-08-11 00:31:18 on dist-test-slave-fd7l"
server_key: "6f6a215e59859682e44238a36b0bc935"
server_key_iv: "415f18204a49e80016354636b84a94a0"
server_key_version: "encryptionkey@0"
I20260811 00:31:18.228796 22076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "e952bc995e4044c289a305da41f7034e"
format_stamp: "Formatted at 2026-08-11 00:31:18 on dist-test-slave-fd7l"
server_key: "6f6a215e59859682e44238a36b0bc935"
server_key_iv: "415f18204a49e80016354636b84a94a0"
server_key_version: "encryptionkey@0"
I20260811 00:31:18.236016 22076 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260811 00:31:18.241895 22095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:18.244140 22076 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:31:18.244490 22076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "e952bc995e4044c289a305da41f7034e"
format_stamp: "Formatted at 2026-08-11 00:31:18 on dist-test-slave-fd7l"
server_key: "6f6a215e59859682e44238a36b0bc935"
server_key_iv: "415f18204a49e80016354636b84a94a0"
server_key_version: "encryptionkey@0"
I20260811 00:31:18.244843 22076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:18.260836 22076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:18.266916 22076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:18.267344 22076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:18.292209 22076 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:45819
I20260811 00:31:18.292241 22154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:45819 every 8 connection(s)
I20260811 00:31:18.294010 22076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:31:18.298982 22155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:18.300926 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 22076
I20260811 00:31:18.301186 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:31:18.302157 2528 external_mini_cluster.cc:1428] Setting key 45400b7473afbca8ce6812894121e31f
I20260811 00:31:18.317889 22155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e: Bootstrap starting.
Aug 11 00:31:18 dist-test-slave-fd7l krb5kdc[22063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:31:18.322873 22155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:18.324285 22155 log.cc:824] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:18.328101 22155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e: No bootstrap required, opened a new log
I20260811 00:31:18.333596 22158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:18.304913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46954 (local address 127.2.120.62:45819)
0811 00:31:18.305984 (+ 1071us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:18.305999 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:18.306053 (+ 54us) server_negotiation.cc:407] Connection header received
0811 00:31:18.308132 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:18.308166 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:18.308900 (+ 734us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:18.309364 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:18.311638 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:18.313509 (+ 1871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:18.315872 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:18.317010 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:18.317216 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:18.323325 (+ 6109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:18.323344 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:18.323366 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:18.323430 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:18.327526 (+ 4096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:18.328795 (+ 1269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:18.328810 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:18.328818 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:18.329054 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:18.329635 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:18.329640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:18.329642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:18.330327 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:18.330789 (+ 462us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:18.331387 (+ 598us) server_negotiation.cc:299] Negotiation successful
0811 00:31:18.331975 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":795,"thread_start_us":499,"threads_started":1}
I20260811 00:31:18.338464 22155 raft_consensus.cc:348] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e952bc995e4044c289a305da41f7034e" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 45819 } }
I20260811 00:31:18.338953 22155 raft_consensus.cc:374] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:18.339085 22155 raft_consensus.cc:729] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e952bc995e4044c289a305da41f7034e, State: Initialized, Role: FOLLOWER
I20260811 00:31:18.339720 22155 consensus_queue.cc:260] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e952bc995e4044c289a305da41f7034e" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 45819 } }
I20260811 00:31:18.340010 22155 raft_consensus.cc:388] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:31:18.340176 22155 raft_consensus.cc:482] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:31:18.340369 22155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:18.342962 22155 raft_consensus.cc:504] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e952bc995e4044c289a305da41f7034e" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 45819 } }
I20260811 00:31:18.343657 22155 leader_election.cc:302] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e952bc995e4044c289a305da41f7034e; no voters:
I20260811 00:31:18.344609 22155 leader_election.cc:288] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:31:18.344864 22161 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:18.345628 22161 raft_consensus.cc:686] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e [term 1 LEADER]: Becoming Leader. State: Replica: e952bc995e4044c289a305da41f7034e, State: Running, Role: LEADER
I20260811 00:31:18.346336 22161 consensus_queue.cc:237] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e952bc995e4044c289a305da41f7034e" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 45819 } }
I20260811 00:31:18.347645 22155 sys_catalog.cc:564] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:31:18.349737 22162 sys_catalog.cc:455] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e952bc995e4044c289a305da41f7034e. Latest consensus state: current_term: 1 leader_uuid: "e952bc995e4044c289a305da41f7034e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e952bc995e4044c289a305da41f7034e" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 45819 } } }
I20260811 00:31:18.349690 22163 sys_catalog.cc:455] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e952bc995e4044c289a305da41f7034e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e952bc995e4044c289a305da41f7034e" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 45819 } } }
I20260811 00:31:18.350205 22162 sys_catalog.cc:458] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:18.350351 22163 sys_catalog.cc:458] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:18.351251 22167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:31:18.357434 22167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:31:18.373158 22167 catalog_manager.cc:1347] Generated new cluster ID: 5bdf4b88e1bf4b26a0ced2c12419f219
I20260811 00:31:18.373458 22167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:31:18.399917 22167 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:31:18.402268 22167 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:31:18.415145 22167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e: Generated new TSK 0
I20260811 00:31:18.416348 22167 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
I20260811 00:31:18.514425 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:45819
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:36021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:31:18.915423 22185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:18.915745 22185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:18.915845 22185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:18.923677 22185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:18.923828 22185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:18.923956 22185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:31:18.934566 22185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:36021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:45819
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:18.936024 22185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:18.937515 22185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:18.953035 22191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:18.953260 22190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:18.953265 22193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:18.954211 22185 server_base.cc:1034] running on GCE node
I20260811 00:31:18.954969 22185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:18.956125 22185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:18.957393 22185 hybrid_clock.cc:648] HybridClock initialized: now 1786408278957333 us; error 58 us; skew 500 ppm
I20260811 00:31:18.957773 22185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:18 dist-test-slave-fd7l krb5kdc[22063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408278, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:18.961515 22185 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:31:18.963183 22185 webserver.cc:460] Webserver started at http://127.2.120.1:46603/ using document root <none> and password file <none>
I20260811 00:31:18.963960 22185 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:18.964082 22185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:18.964387 22185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:18.968066 22185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "05f1b493758844d889c4100284c90d8c"
format_stamp: "Formatted at 2026-08-11 00:31:18 on dist-test-slave-fd7l"
server_key: "66fdd497eb8d704a7d80e455d99dea5d"
server_key_iv: "c3e89e86b6899cf5db9365da3891e609"
server_key_version: "encryptionkey@0"
I20260811 00:31:18.969070 22185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "05f1b493758844d889c4100284c90d8c"
format_stamp: "Formatted at 2026-08-11 00:31:18 on dist-test-slave-fd7l"
server_key: "66fdd497eb8d704a7d80e455d99dea5d"
server_key_iv: "c3e89e86b6899cf5db9365da3891e609"
server_key_version: "encryptionkey@0"
I20260811 00:31:18.975416 22185 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 00:31:18.980340 22201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:18.982587 22185 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 00:31:18.982868 22185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "05f1b493758844d889c4100284c90d8c"
format_stamp: "Formatted at 2026-08-11 00:31:18 on dist-test-slave-fd7l"
server_key: "66fdd497eb8d704a7d80e455d99dea5d"
server_key_iv: "c3e89e86b6899cf5db9365da3891e609"
server_key_version: "encryptionkey@0"
I20260811 00:31:18.983127 22185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:19.005430 22185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:19.011621 22185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:19.011930 22185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:19.013190 22185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:19.014693 22185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:19.014783 22185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:19.014879 22185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:19.014928 22185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:19.051615 22185 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:38821
I20260811 00:31:19.051725 22330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:38821 every 8 connection(s)
I20260811 00:31:19.053134 22185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:31:19.062431 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 22185
I20260811 00:31:19.062662 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:31:19.063503 2528 external_mini_cluster.cc:1428] Setting key 4cd7febdc1a75a6057aace7ff3b7c077
Aug 11 00:31:19 dist-test-slave-fd7l krb5kdc[22063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408278, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:31:19.078946 22333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:19.056470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:40971 (local address 127.2.120.62:45819)
0811 00:31:19.057047 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:19.057052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:19.057897 (+ 845us) server_negotiation.cc:407] Connection header received
0811 00:31:19.059841 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:19.059846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:19.059962 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:19.060095 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:19.061837 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:19.063091 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:19.065719 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:19.066357 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:19.066557 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:19.071806 (+ 5249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:19.071813 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:19.071816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:19.071858 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:19.075045 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:19.076049 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:19.076055 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:19.076058 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:19.076204 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:19.076840 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:19.076851 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:19.076853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:19.077164 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:19.077351 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:19.078248 (+ 897us) server_negotiation.cc:299] Negotiation successful
0811 00:31:19.078516 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":284,"threads_started":1}
I20260811 00:31:19.080050 22334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:19.056941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:45819 (local address 127.2.120.1:40971)
0811 00:31:19.057711 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:19.057751 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:19.059525 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:19.060365 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:19.060374 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:19.060988 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:19.061616 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:19.061640 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:19.063335 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:19.063340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:19.065430 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:19.065450 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:19.066701 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:19.066711 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:19.067003 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:19.067883 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:31:19.067912 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:19.071443 (+ 3531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:19.075277 (+ 3834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:19.075289 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:19.075312 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:19.075863 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:19.076349 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:19.076363 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:19.076368 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:19.076691 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:19.077323 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:19.077333 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:19.077788 (+ 455us) client_negotiation.cc:769] Sending connection context
0811 00:31:19.078105 (+ 317us) client_negotiation.cc:240] Negotiation successful
0811 00:31:19.078746 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":306,"threads_started":1}
I20260811 00:31:19.082301 22331 heartbeater.cc:347] Connected to a master server at 127.2.120.62:45819
I20260811 00:31:19.082706 22331 heartbeater.cc:464] Registering TS with master...
I20260811 00:31:19.083705 22331 heartbeater.cc:511] Master 127.2.120.62:45819 requested a full tablet report, sending...
I20260811 00:31:19.086551 22114 ts_manager.cc:194] Registered new tserver with Master: 05f1b493758844d889c4100284c90d8c (127.2.120.1:38821)
I20260811 00:31:19.088402 22114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:40971
WARNING: no policy specified for kudu/127.2.120.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
I20260811 00:31:19.142045 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:45819
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:36021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:31:19.557271 22337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:19.557638 22337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:19.557744 22337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:19.565835 22337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:19.565984 22337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:19.566154 22337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:31:19.579953 22337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:36021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:45819
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:19.581691 22337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:19.583324 22337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:19.599654 22347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:19.599714 22344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:19.599748 22343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:19.600759 22337 server_base.cc:1034] running on GCE node
I20260811 00:31:19.601588 22337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:19.602816 22337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:19.604368 22337 hybrid_clock.cc:648] HybridClock initialized: now 1786408279604238 us; error 106 us; skew 500 ppm
I20260811 00:31:19.604848 22337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:19 dist-test-slave-fd7l krb5kdc[22063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408279, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:19.609880 22337 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:31:19.611955 22337 webserver.cc:460] Webserver started at http://127.2.120.2:40523/ using document root <none> and password file <none>
I20260811 00:31:19.612937 22337 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:19.613121 22337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:19.613487 22337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:19.616941 22337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "8e4bcc1839b14559b76fc472b1e45865"
format_stamp: "Formatted at 2026-08-11 00:31:19 on dist-test-slave-fd7l"
server_key: "db3a306eca0aa26431822025c9a56303"
server_key_iv: "49ca4e28b03c4150c5c5ccb06eadae91"
server_key_version: "encryptionkey@0"
I20260811 00:31:19.617931 22337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "8e4bcc1839b14559b76fc472b1e45865"
format_stamp: "Formatted at 2026-08-11 00:31:19 on dist-test-slave-fd7l"
server_key: "db3a306eca0aa26431822025c9a56303"
server_key_iv: "49ca4e28b03c4150c5c5ccb06eadae91"
server_key_version: "encryptionkey@0"
I20260811 00:31:19.624482 22337 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 00:31:19.628921 22354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:19.630952 22337 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260811 00:31:19.631222 22337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "8e4bcc1839b14559b76fc472b1e45865"
format_stamp: "Formatted at 2026-08-11 00:31:19 on dist-test-slave-fd7l"
server_key: "db3a306eca0aa26431822025c9a56303"
server_key_iv: "49ca4e28b03c4150c5c5ccb06eadae91"
server_key_version: "encryptionkey@0"
I20260811 00:31:19.631505 22337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:19.669222 22337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:19.674885 22337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:19.675262 22337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:19.676806 22337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:19.678642 22337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:19.678777 22337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:19.678973 22337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:19.679054 22337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:19.719878 22337 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:46215
I20260811 00:31:19.720029 22484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:46215 every 8 connection(s)
I20260811 00:31:19.721484 22337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:31:19.730841 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 22337
I20260811 00:31:19.731245 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:31:19.731950 2528 external_mini_cluster.cc:1428] Setting key f1101a44e020884e1ba80a0fe38f4929
Aug 11 00:31:19 dist-test-slave-fd7l krb5kdc[22063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408279, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:31:19.746613 22487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:19.724758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:53467 (local address 127.2.120.62:45819)
0811 00:31:19.725305 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:19.725310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:19.726162 (+ 852us) server_negotiation.cc:407] Connection header received
0811 00:31:19.728253 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:19.728259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:19.728383 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:19.728526 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:19.730426 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:19.731364 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:19.733391 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:19.734108 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:19.734289 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:19.739098 (+ 4809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:19.739105 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:19.739109 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:19.739153 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:19.742447 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:19.743511 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:19.743517 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:19.743519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:19.743668 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:19.744293 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:19.744304 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:19.744306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:19.744730 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:19.744918 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:19.745845 (+ 927us) server_negotiation.cc:299] Negotiation successful
0811 00:31:19.746171 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":299,"threads_started":1}
I20260811 00:31:19.748142 22488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:19.725193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:45819 (local address 127.2.120.2:53467)
0811 00:31:19.725962 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:19.726002 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:19.727893 (+ 1891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:19.728860 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:19.728872 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:19.729521 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:19.730192 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:19.730214 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:19.731600 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:19.731606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:19.733197 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:19.733213 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:19.734415 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:19.734424 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:19.734696 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:19.735545 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:31:19.735579 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:19.738725 (+ 3146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:19.742684 (+ 3959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:19.742697 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:19.742726 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:19.743311 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:19.743813 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:19.743819 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:19.743823 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:19.744137 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:19.744891 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:19.744902 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:19.745375 (+ 473us) client_negotiation.cc:769] Sending connection context
0811 00:31:19.745733 (+ 358us) client_negotiation.cc:240] Negotiation successful
0811 00:31:19.746415 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":317,"threads_started":1}
I20260811 00:31:19.750486 22485 heartbeater.cc:347] Connected to a master server at 127.2.120.62:45819
I20260811 00:31:19.750937 22485 heartbeater.cc:464] Registering TS with master...
I20260811 00:31:19.751883 22485 heartbeater.cc:511] Master 127.2.120.62:45819 requested a full tablet report, sending...
I20260811 00:31:19.754359 22114 ts_manager.cc:194] Registered new tserver with Master: 8e4bcc1839b14559b76fc472b1e45865 (127.2.120.2:46215)
I20260811 00:31:19.755668 22114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:53467
WARNING: no policy specified for kudu/127.2.120.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
I20260811 00:31:19.818434 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:45819
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:36021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:31:20.092912 22331 heartbeater.cc:503] Master 127.2.120.62:45819 was elected leader, sending a full tablet report...
W20260811 00:31:20.201457 22491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:20.201794 22491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:20.201963 22491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:20.210099 22491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:20.210243 22491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:20.210444 22491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:31:20.221266 22491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:36021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:45819
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:20.222826 22491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:20.224309 22491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:20.241719 22497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:20.241763 22500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:20.241714 22498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:20.243167 22491 server_base.cc:1034] running on GCE node
I20260811 00:31:20.243945 22491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:20.245234 22491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:20.246546 22491 hybrid_clock.cc:648] HybridClock initialized: now 1786408280246487 us; error 52 us; skew 500 ppm
I20260811 00:31:20.246974 22491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:20 dist-test-slave-fd7l krb5kdc[22063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408280, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:20.251477 22491 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:31:20.253718 22491 webserver.cc:460] Webserver started at http://127.2.120.3:43579/ using document root <none> and password file <none>
I20260811 00:31:20.254779 22491 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:20.254999 22491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:20.255414 22491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:20.259055 22491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "5c5378df69234b65bcb9b4182222470e"
format_stamp: "Formatted at 2026-08-11 00:31:20 on dist-test-slave-fd7l"
server_key: "125649ec12377d81dfbac79d181dc2d5"
server_key_iv: "439cfc5cd0eff66cc4ebd96e56b8b850"
server_key_version: "encryptionkey@0"
I20260811 00:31:20.260068 22491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "5c5378df69234b65bcb9b4182222470e"
format_stamp: "Formatted at 2026-08-11 00:31:20 on dist-test-slave-fd7l"
server_key: "125649ec12377d81dfbac79d181dc2d5"
server_key_iv: "439cfc5cd0eff66cc4ebd96e56b8b850"
server_key_version: "encryptionkey@0"
I20260811 00:31:20.266440 22491 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 00:31:20.270573 22507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:20.272406 22491 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 00:31:20.272699 22491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "5c5378df69234b65bcb9b4182222470e"
format_stamp: "Formatted at 2026-08-11 00:31:20 on dist-test-slave-fd7l"
server_key: "125649ec12377d81dfbac79d181dc2d5"
server_key_iv: "439cfc5cd0eff66cc4ebd96e56b8b850"
server_key_version: "encryptionkey@0"
I20260811 00:31:20.272956 22491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:20.293766 22491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:20.299444 22491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:20.299782 22491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:20.301079 22491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:20.303156 22491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:20.303243 22491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:20.303341 22491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:20.303386 22491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:20.341622 22491 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:41653
I20260811 00:31:20.341677 22636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:41653 every 8 connection(s)
I20260811 00:31:20.343060 22491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:31:20.344342 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 22491
I20260811 00:31:20.344588 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:31:20.345247 2528 external_mini_cluster.cc:1428] Setting key 387c63c6381d57abf590edb73237e8ff
Aug 11 00:31:20 dist-test-slave-fd7l krb5kdc[22063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408280, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:31:20.367640 22639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:20.346113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:46197 (local address 127.2.120.62:45819)
0811 00:31:20.346710 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:20.346716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:20.347980 (+ 1264us) server_negotiation.cc:407] Connection header received
0811 00:31:20.350422 (+ 2442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:20.350427 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:20.350514 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:20.350616 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:20.352464 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.353423 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.355120 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.355550 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.355678 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:20.359856 (+ 4178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:20.359862 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:20.359865 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:20.359907 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:20.363482 (+ 3575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:20.364458 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:20.364468 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:20.364472 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:20.364674 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:20.365244 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:20.365252 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:20.365257 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:20.365589 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:20.365878 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:20.366730 (+ 852us) server_negotiation.cc:299] Negotiation successful
0811 00:31:20.367011 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":275,"threads_started":1}
I20260811 00:31:20.368961 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:20.346821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:45819 (local address 127.2.120.3:46197)
0811 00:31:20.347774 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:20.347840 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:20.350109 (+ 2269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:20.350959 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:20.350969 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:20.351577 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:20.352237 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:20.352258 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.353599 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.353604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:20.354941 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:20.354956 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.355854 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.355861 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:20.356085 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:20.356948 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:31:20.356981 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:20.359554 (+ 2573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:20.363716 (+ 4162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:20.363726 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:20.363743 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:20.364273 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:20.364859 (+ 586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:20.364866 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:20.364870 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:20.365103 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:20.365754 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:20.365765 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:20.366289 (+ 524us) client_negotiation.cc:769] Sending connection context
0811 00:31:20.366639 (+ 350us) client_negotiation.cc:240] Negotiation successful
0811 00:31:20.367296 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":360,"threads_started":1}
I20260811 00:31:20.370687 22637 heartbeater.cc:347] Connected to a master server at 127.2.120.62:45819
I20260811 00:31:20.371107 22637 heartbeater.cc:464] Registering TS with master...
I20260811 00:31:20.372040 22637 heartbeater.cc:511] Master 127.2.120.62:45819 requested a full tablet report, sending...
I20260811 00:31:20.374274 22114 ts_manager.cc:194] Registered new tserver with Master: 5c5378df69234b65bcb9b4182222470e (127.2.120.3:41653)
I20260811 00:31:20.375249 22114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:46197
I20260811 00:31:20.375814 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:31:20.400998 22639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:20.382107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46956 (local address 127.2.120.62:45819)
0811 00:31:20.382330 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:20.382335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:20.382709 (+ 374us) server_negotiation.cc:407] Connection header received
0811 00:31:20.383098 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:20.383103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:20.383178 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:20.383272 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:20.385025 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.385804 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.387553 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.388200 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.388416 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:20.391331 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:20.391339 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:20.391343 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:20.391398 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:20.395832 (+ 4434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:20.397895 (+ 2063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:20.397901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:20.397904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:20.398111 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:20.398717 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:20.398721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:20.398724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:20.399201 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:20.399428 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:20.400185 (+ 757us) server_negotiation.cc:299] Negotiation successful
0811 00:31:20.400460 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 00:31:20.406922 22114 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46956:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:31:20.415616 22114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:31:20.445140 22654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:20.432153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48946 (local address 127.2.120.3:41653)
0811 00:31:20.433249 (+ 1096us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:20.433258 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:20.436231 (+ 2973us) server_negotiation.cc:407] Connection header received
0811 00:31:20.436404 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:20.436416 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:20.436675 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:20.436863 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:20.437866 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.439498 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.442453 (+ 2955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.443724 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.443863 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:20.444093 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:20.444604 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 00:31:20.444777 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":893,"thread_start_us":685,"threads_started":1}
I20260811 00:31:20.445134 22655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:20.431469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:41653 (local address 127.0.0.1:48946)
0811 00:31:20.436102 (+ 4633us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:20.436121 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:20.436244 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:20.437117 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:20.437123 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:20.437146 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:20.437678 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:20.437693 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.439674 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.439678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:20.442240 (+ 2562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:20.442258 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.444231 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.444235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:20.444377 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:20.444403 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 00:31:20.444520 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 00:31:20.444693 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4506,"mutex_wait_us":4,"thread_start_us":4045,"threads_started":1}
I20260811 00:31:20.445317 22651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:20.431920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46580 (local address 127.2.120.1:38821)
0811 00:31:20.432638 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:20.432649 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:20.435633 (+ 2984us) server_negotiation.cc:407] Connection header received
0811 00:31:20.436083 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:20.436095 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:20.436430 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:20.436706 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:20.437765 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.438958 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.441728 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.442986 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.443123 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:20.443365 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:20.444823 (+ 1458us) server_negotiation.cc:299] Negotiation successful
0811 00:31:20.444987 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":351,"threads_started":1}
I20260811 00:31:20.446043 22652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:20.432013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56758 (local address 127.2.120.2:46215)
0811 00:31:20.433601 (+ 1588us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:20.433613 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:20.436955 (+ 3342us) server_negotiation.cc:407] Connection header received
0811 00:31:20.437064 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:20.437071 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:20.437262 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:20.437457 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:20.438207 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.439790 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.442508 (+ 2718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.443965 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.444083 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:20.444255 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:20.445667 (+ 1412us) server_negotiation.cc:299] Negotiation successful
0811 00:31:20.445798 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1374,"thread_start_us":598,"threads_started":1}
I20260811 00:31:20.446341 22653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:20.431511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:46215 (local address 127.0.0.1:56758)
0811 00:31:20.436687 (+ 5176us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:20.436706 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:20.436835 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:20.437463 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:20.437467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:20.437499 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:20.438000 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:20.438013 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.439966 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.439974 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:20.442341 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:20.442355 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.444414 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.444420 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:20.444638 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:20.444686 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 00:31:20.444780 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 00:31:20.444915 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5051,"mutex_wait_us":101,"thread_start_us":4888,"threads_started":1}
I20260811 00:31:20.444680 22650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:20.430972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:38821 (local address 127.0.0.1:46580)
0811 00:31:20.435310 (+ 4338us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:20.435494 (+ 184us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:20.435807 (+ 313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:20.437000 (+ 1193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:20.437007 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:20.437039 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:20.437574 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:20.437594 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.439192 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.439197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:20.441547 (+ 2350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:20.441560 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.443514 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.443520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:20.443716 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:20.443800 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 00:31:20.443940 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 00:31:20.444158 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":149,"thread_start_us":459,"threads_started":1}
I20260811 00:31:20.450114 22564 tablet_service.cc:1467] Processing CreateTablet for tablet 9a66f57d0f4a4994aa7876b0abd307e9 (DEFAULT_TABLE table=test-table [id=d66b68200ab54ab7914bb3c96cf10d01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:31:20.450141 22410 tablet_service.cc:1467] Processing CreateTablet for tablet 9a66f57d0f4a4994aa7876b0abd307e9 (DEFAULT_TABLE table=test-table [id=d66b68200ab54ab7914bb3c96cf10d01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:31:20.451195 22257 tablet_service.cc:1467] Processing CreateTablet for tablet 9a66f57d0f4a4994aa7876b0abd307e9 (DEFAULT_TABLE table=test-table [id=d66b68200ab54ab7914bb3c96cf10d01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:31:20.451817 22564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a66f57d0f4a4994aa7876b0abd307e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:20.451859 22410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a66f57d0f4a4994aa7876b0abd307e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:20.452904 22257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a66f57d0f4a4994aa7876b0abd307e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:20.464884 22656 tablet_bootstrap.cc:492] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865: Bootstrap starting.
I20260811 00:31:20.468571 22657 tablet_bootstrap.cc:492] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e: Bootstrap starting.
I20260811 00:31:20.469739 22656 tablet_bootstrap.cc:654] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:20.471277 22656 log.cc:824] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:20.472445 22658 tablet_bootstrap.cc:492] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c: Bootstrap starting.
I20260811 00:31:20.472947 22657 tablet_bootstrap.cc:654] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:20.474699 22657 log.cc:824] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:20.476198 22658 tablet_bootstrap.cc:654] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:20.477833 22658 log.cc:824] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:20.478626 22657 tablet_bootstrap.cc:492] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e: No bootstrap required, opened a new log
I20260811 00:31:20.478947 22657 ts_tablet_manager.cc:1397] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 00:31:20.479590 22656 tablet_bootstrap.cc:492] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865: No bootstrap required, opened a new log
I20260811 00:31:20.479874 22656 ts_tablet_manager.cc:1397] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.007s
I20260811 00:31:20.481002 22658 tablet_bootstrap.cc:492] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c: No bootstrap required, opened a new log
I20260811 00:31:20.481276 22658 ts_tablet_manager.cc:1397] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 00:31:20.489070 22658 raft_consensus.cc:348] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f1b493758844d889c4100284c90d8c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38821 } } peers { permanent_uuid: "5c5378df69234b65bcb9b4182222470e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41653 } } peers { permanent_uuid: "8e4bcc1839b14559b76fc472b1e45865" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46215 } }
I20260811 00:31:20.489619 22658 raft_consensus.cc:374] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:20.489738 22658 raft_consensus.cc:729] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05f1b493758844d889c4100284c90d8c, State: Initialized, Role: FOLLOWER
I20260811 00:31:20.490180 22657 raft_consensus.cc:348] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f1b493758844d889c4100284c90d8c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38821 } } peers { permanent_uuid: "5c5378df69234b65bcb9b4182222470e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41653 } } peers { permanent_uuid: "8e4bcc1839b14559b76fc472b1e45865" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46215 } }
I20260811 00:31:20.490396 22658 consensus_queue.cc:260] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f1b493758844d889c4100284c90d8c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38821 } } peers { permanent_uuid: "5c5378df69234b65bcb9b4182222470e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41653 } } peers { permanent_uuid: "8e4bcc1839b14559b76fc472b1e45865" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46215 } }
I20260811 00:31:20.490672 22657 raft_consensus.cc:374] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:20.490777 22657 raft_consensus.cc:729] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c5378df69234b65bcb9b4182222470e, State: Initialized, Role: FOLLOWER
I20260811 00:31:20.491496 22657 consensus_queue.cc:260] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f1b493758844d889c4100284c90d8c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38821 } } peers { permanent_uuid: "5c5378df69234b65bcb9b4182222470e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41653 } } peers { permanent_uuid: "8e4bcc1839b14559b76fc472b1e45865" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46215 } }
I20260811 00:31:20.491572 22656 raft_consensus.cc:348] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f1b493758844d889c4100284c90d8c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38821 } } peers { permanent_uuid: "5c5378df69234b65bcb9b4182222470e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41653 } } peers { permanent_uuid: "8e4bcc1839b14559b76fc472b1e45865" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46215 } }
I20260811 00:31:20.491986 22656 raft_consensus.cc:374] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:20.492120 22656 raft_consensus.cc:729] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e4bcc1839b14559b76fc472b1e45865, State: Initialized, Role: FOLLOWER
I20260811 00:31:20.492408 22658 ts_tablet_manager.cc:1428] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260811 00:31:20.492982 22656 consensus_queue.cc:260] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f1b493758844d889c4100284c90d8c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38821 } } peers { permanent_uuid: "5c5378df69234b65bcb9b4182222470e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41653 } } peers { permanent_uuid: "8e4bcc1839b14559b76fc472b1e45865" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46215 } }
I20260811 00:31:20.494374 22485 heartbeater.cc:503] Master 127.2.120.62:45819 was elected leader, sending a full tablet report...
I20260811 00:31:20.497156 22656 ts_tablet_manager.cc:1428] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865: Time spent starting tablet: real 0.017s user 0.010s sys 0.006s
I20260811 00:31:20.498315 22637 heartbeater.cc:503] Master 127.2.120.62:45819 was elected leader, sending a full tablet report...
I20260811 00:31:20.499943 22657 ts_tablet_manager.cc:1428] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e: Time spent starting tablet: real 0.021s user 0.017s sys 0.004s
W20260811 00:31:20.556259 22332 tablet.cc:2369] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:31:20.594825 22638 tablet.cc:2369] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:31:20.686726 22665 raft_consensus.cc:482] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:31:20.687211 22665 raft_consensus.cc:504] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f1b493758844d889c4100284c90d8c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38821 } } peers { permanent_uuid: "5c5378df69234b65bcb9b4182222470e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41653 } } peers { permanent_uuid: "8e4bcc1839b14559b76fc472b1e45865" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46215 } }
I20260811 00:31:20.689431 22665 leader_election.cc:288] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05f1b493758844d889c4100284c90d8c (127.2.120.1:38821), 8e4bcc1839b14559b76fc472b1e45865 (127.2.120.2:46215)
I20260811 00:31:20.696493 22651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:20.689802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:60995 (local address 127.2.120.1:38821)
0811 00:31:20.689989 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:20.689995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:20.690334 (+ 339us) server_negotiation.cc:407] Connection header received
0811 00:31:20.690417 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:20.690420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:20.690487 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:20.690602 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:20.691065 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.692212 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.694261 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.695523 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.695636 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:20.695721 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:20.696146 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 00:31:20.696272 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 00:31:20.696707 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:20.689632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:38821 (local address 127.2.120.3:60995)
0811 00:31:20.690000 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:20.690022 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:20.690215 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:20.690629 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:20.690634 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:20.690666 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:20.690860 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:20.690869 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.692365 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.692371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:20.694051 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:20.694065 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.695720 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.695727 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:20.695936 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:20.695973 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 00:31:20.696088 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 00:31:20.696244 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190}
I20260811 00:31:20.697886 22652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:20.690034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:42937 (local address 127.2.120.2:46215)
0811 00:31:20.690286 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:20.690293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:20.690978 (+ 685us) server_negotiation.cc:407] Connection header received
0811 00:31:20.691175 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:20.691179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:20.691238 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:20.691308 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:20.692033 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.693104 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.695403 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.696840 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.697020 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:20.697141 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:20.697482 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 00:31:20.697623 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 00:31:20.697974 22667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:20.689901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:46215 (local address 127.2.120.3:42937)
0811 00:31:20.690834 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:20.690851 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:20.691032 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:20.691374 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:20.691380 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:20.691409 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:20.691832 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:20.691845 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.693291 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.693296 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:20.695143 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:20.695163 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.697146 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.697154 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:20.697301 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:20.697326 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 00:31:20.697454 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 00:31:20.697623 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"thread_start_us":507,"threads_started":1}
I20260811 00:31:20.697959 22279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a66f57d0f4a4994aa7876b0abd307e9" candidate_uuid: "5c5378df69234b65bcb9b4182222470e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05f1b493758844d889c4100284c90d8c" is_pre_election: true
I20260811 00:31:20.698596 22279 raft_consensus.cc:2454] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c5378df69234b65bcb9b4182222470e in term 0.
I20260811 00:31:20.699210 22430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a66f57d0f4a4994aa7876b0abd307e9" candidate_uuid: "5c5378df69234b65bcb9b4182222470e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e4bcc1839b14559b76fc472b1e45865" is_pre_election: true
I20260811 00:31:20.699514 22512 leader_election.cc:302] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05f1b493758844d889c4100284c90d8c, 5c5378df69234b65bcb9b4182222470e; no voters:
I20260811 00:31:20.699777 22430 raft_consensus.cc:2454] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c5378df69234b65bcb9b4182222470e in term 0.
I20260811 00:31:20.700140 22665 raft_consensus.cc:2781] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:31:20.700321 22665 raft_consensus.cc:482] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:31:20.700809 22665 raft_consensus.cc:3037] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:20.703282 22665 raft_consensus.cc:504] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f1b493758844d889c4100284c90d8c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38821 } } peers { permanent_uuid: "5c5378df69234b65bcb9b4182222470e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41653 } } peers { permanent_uuid: "8e4bcc1839b14559b76fc472b1e45865" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46215 } }
I20260811 00:31:20.704092 22665 leader_election.cc:288] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [CANDIDATE]: Term 1 election: Requested vote from peers 05f1b493758844d889c4100284c90d8c (127.2.120.1:38821), 8e4bcc1839b14559b76fc472b1e45865 (127.2.120.2:46215)
I20260811 00:31:20.704694 22279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a66f57d0f4a4994aa7876b0abd307e9" candidate_uuid: "5c5378df69234b65bcb9b4182222470e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05f1b493758844d889c4100284c90d8c"
I20260811 00:31:20.704731 22430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a66f57d0f4a4994aa7876b0abd307e9" candidate_uuid: "5c5378df69234b65bcb9b4182222470e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e4bcc1839b14559b76fc472b1e45865"
I20260811 00:31:20.704934 22279 raft_consensus.cc:3037] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:20.704954 22430 raft_consensus.cc:3037] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:20.707283 22430 raft_consensus.cc:2454] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c5378df69234b65bcb9b4182222470e in term 1.
I20260811 00:31:20.707281 22279 raft_consensus.cc:2454] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c5378df69234b65bcb9b4182222470e in term 1.
I20260811 00:31:20.707859 22510 leader_election.cc:302] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c5378df69234b65bcb9b4182222470e, 8e4bcc1839b14559b76fc472b1e45865; no voters:
I20260811 00:31:20.708297 22665 raft_consensus.cc:2781] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:20.708878 22665 raft_consensus.cc:686] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [term 1 LEADER]: Becoming Leader. State: Replica: 5c5378df69234b65bcb9b4182222470e, State: Running, Role: LEADER
I20260811 00:31:20.709438 22665 consensus_queue.cc:237] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f1b493758844d889c4100284c90d8c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38821 } } peers { permanent_uuid: "5c5378df69234b65bcb9b4182222470e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41653 } } peers { permanent_uuid: "8e4bcc1839b14559b76fc472b1e45865" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46215 } }
I20260811 00:31:20.715075 22113 catalog_manager.cc:5495] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c5378df69234b65bcb9b4182222470e (127.2.120.3). New cstate: current_term: 1 leader_uuid: "5c5378df69234b65bcb9b4182222470e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05f1b493758844d889c4100284c90d8c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c5378df69234b65bcb9b4182222470e" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 41653 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e4bcc1839b14559b76fc472b1e45865" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46215 } health_report { overall_health: UNKNOWN } } }
W20260811 00:31:20.724045 22486 tablet.cc:2369] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:31:20.751138 22654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:20.744819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48952 (local address 127.2.120.3:41653)
0811 00:31:20.745031 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:20.745036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:20.745175 (+ 139us) server_negotiation.cc:407] Connection header received
0811 00:31:20.745440 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:20.745445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:20.745533 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:20.745707 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:31:20.746083 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.746853 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.748617 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.749111 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.749222 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:20.749853 (+ 631us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:31:20.750432 (+ 579us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:31:20.750631 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:20.750747 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 00:31:20.750848 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 00:31:20.760697 22279 raft_consensus.cc:1261] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c [term 1 FOLLOWER]: Refusing update from remote peer 5c5378df69234b65bcb9b4182222470e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:31:20.760700 22430 raft_consensus.cc:1261] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865 [term 1 FOLLOWER]: Refusing update from remote peer 5c5378df69234b65bcb9b4182222470e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:31:20.762065 22668 consensus_queue.cc:1035] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e4bcc1839b14559b76fc472b1e45865" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 46215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:31:20.762492 22665 consensus_queue.cc:1035] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [LEADER]: Connected to new peer: Peer: permanent_uuid: "05f1b493758844d889c4100284c90d8c" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 38821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:31:20.777328 22671 mvcc.cc:205] Tried to move back new op lower bound from 7317128317982646272 to 7317128317790535680. Current Snapshot: MvccSnapshot[applied={T|T < 7317128317982646272}]
I20260811 00:31:20.794683 22114 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:31:20.794999 22114 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:31:20.800381 22114 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e952bc995e4044c289a305da41f7034e: Sending DeleteTablet for 3 replicas of tablet 9a66f57d0f4a4994aa7876b0abd307e9
I20260811 00:31:20.802263 22257 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a66f57d0f4a4994aa7876b0abd307e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:31:20 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:46580
I20260811 00:31:20.802258 22564 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a66f57d0f4a4994aa7876b0abd307e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:31:20 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:48946
I20260811 00:31:20.802392 22410 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a66f57d0f4a4994aa7876b0abd307e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:31:20 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:56758
I20260811 00:31:20.808934 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 22185
I20260811 00:31:20.839915 22684 tablet_replica.cc:330] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e: stopping tablet replica
I20260811 00:31:20.840914 22684 raft_consensus.cc:2229] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:31:20.847059 22684 raft_consensus.cc:2258] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:31:20.853443 22684 ts_tablet_manager.cc:1905] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:31:20.854403 22685 tablet_replica.cc:330] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865: stopping tablet replica
I20260811 00:31:20.854677 22683 tablet_replica.cc:330] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c: stopping tablet replica
I20260811 00:31:20.855064 22683 raft_consensus.cc:2229] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:31:20.855065 22685 raft_consensus.cc:2229] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:31:20.855635 22683 raft_consensus.cc:2258] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:31:20.855643 22685 raft_consensus.cc:2258] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:31:20.857815 22683 ts_tablet_manager.cc:1905] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:31:20.859350 22685 ts_tablet_manager.cc:1905] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 00:31:20 dist-test-slave-fd7l krb5kdc[22063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:31:20.863780 22684 ts_tablet_manager.cc:1918] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:31:20.864023 22684 log.cc:1196] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/9a66f57d0f4a4994aa7876b0abd307e9
I20260811 00:31:20.864642 22684 ts_tablet_manager.cc:1939] T 9a66f57d0f4a4994aa7876b0abd307e9 P 5c5378df69234b65bcb9b4182222470e: Deleting consensus metadata
I20260811 00:31:20.867002 22097 catalog_manager.cc:4841] TS 5c5378df69234b65bcb9b4182222470e (127.2.120.3:41653): tablet 9a66f57d0f4a4994aa7876b0abd307e9 (table test-table [id=d66b68200ab54ab7914bb3c96cf10d01]) successfully deleted
I20260811 00:31:20.871672 22683 ts_tablet_manager.cc:1918] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:31:20.871706 22685 ts_tablet_manager.cc:1918] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:31:20.871896 22685 log.cc:1196] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/9a66f57d0f4a4994aa7876b0abd307e9
I20260811 00:31:20.872428 22651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:20.810726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46596 (local address 127.2.120.1:38821)
0811 00:31:20.810986 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:20.810997 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:20.835680 (+ 24683us) server_negotiation.cc:407] Connection header received
0811 00:31:20.836096 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:20.836102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:20.836275 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:20.836386 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:20.841411 (+ 5025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.843244 (+ 1833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.847838 (+ 4594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:20.848867 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:20.849045 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:20.863658 (+ 14613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:20.863683 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:20.863704 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:20.863761 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:20.868116 (+ 4355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:20.869305 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:20.869315 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:20.869319 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:20.869525 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:20.870056 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:20.870061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:20.870065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:20.870584 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:20.870901 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:20.871684 (+ 783us) server_negotiation.cc:299] Negotiation successful
0811 00:31:20.871992 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 00:31:20.872555 22685 ts_tablet_manager.cc:1939] T 9a66f57d0f4a4994aa7876b0abd307e9 P 8e4bcc1839b14559b76fc472b1e45865: Deleting consensus metadata
I20260811 00:31:20.872081 22683 log.cc:1196] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/9a66f57d0f4a4994aa7876b0abd307e9
I20260811 00:31:20.873776 22683 ts_tablet_manager.cc:1939] T 9a66f57d0f4a4994aa7876b0abd307e9 P 05f1b493758844d889c4100284c90d8c: Deleting consensus metadata
I20260811 00:31:20.873893 22326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:20.874851 22098 catalog_manager.cc:4841] TS 8e4bcc1839b14559b76fc472b1e45865 (127.2.120.2:46215): tablet 9a66f57d0f4a4994aa7876b0abd307e9 (table test-table [id=d66b68200ab54ab7914bb3c96cf10d01]) successfully deleted
I20260811 00:31:20.970211 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 22185
I20260811 00:31:20.970290 22099 catalog_manager.cc:4841] TS 05f1b493758844d889c4100284c90d8c (127.2.120.1:38821): tablet 9a66f57d0f4a4994aa7876b0abd307e9 (table test-table [id=d66b68200ab54ab7914bb3c96cf10d01]) successfully deleted
I20260811 00:31:20.999890 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 22337
Aug 11 00:31:21 dist-test-slave-fd7l krb5kdc[22063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.2@KRBTEST.COM
I20260811 00:31:21.020586 22652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:21.001217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56760 (local address 127.2.120.2:46215)
0811 00:31:21.001414 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:21.001420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:21.001444 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:31:21.001632 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:21.001638 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:21.001730 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:21.001854 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:21.003209 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:21.004352 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:21.006097 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:21.006813 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:21.006987 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:21.012176 (+ 5189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:21.012203 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:21.012229 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:21.012294 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:21.016771 (+ 4477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:21.017982 (+ 1211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:21.017994 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:21.017999 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:21.018189 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:21.018678 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:21.018686 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:21.018691 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:21.019239 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:21.019575 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:21.019892 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 00:31:21.020155 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:31:21.021761 22480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:21.119872 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 22337
I20260811 00:31:21.144372 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 22491
Aug 11 00:31:21 dist-test-slave-fd7l krb5kdc[22063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.3@KRBTEST.COM
I20260811 00:31:21.163435 22654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:21.145449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48968 (local address 127.2.120.3:41653)
0811 00:31:21.145668 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:21.145676 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:21.145700 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:31:21.145778 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:21.145783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:21.145869 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:21.146008 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:21.147382 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:21.148415 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:21.150377 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:21.151008 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:21.151203 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:21.156247 (+ 5044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:21.156262 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:21.156279 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:21.156338 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:21.159892 (+ 3554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:21.160933 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:21.160943 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:21.160946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:21.161127 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:21.161574 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:21.161580 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:21.161583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:21.162011 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:21.162420 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:21.162764 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 00:31:21.163028 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 00:31:21.164194 22632 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:21.299360 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 22491
I20260811 00:31:21.325152 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 22076
I20260811 00:31:21.326900 22149 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:21.449141 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 22076
2026-08-11T00:31:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5322 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T00:31:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:31:21Z Disabled control of system clock
I20260811 00:31:21.564069 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:40369
--webserver_interface=127.2.120.62
--webserver_port=0
--builtin_ntp_servers=127.2.120.20:46659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:40369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 00:31:21.959223 22694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:21.959569 22694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:21.959730 22694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:21.967504 22694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:31:21.967743 22694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:21.967830 22694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:21.967912 22694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:31:21.967959 22694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:31:21.977802 22694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:46659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:40369
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:40369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:21.979476 22694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:21.981478 22694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:21.994975 22701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:21.995097 22704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:21.995069 22700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:21.996714 22694 server_base.cc:1034] running on GCE node
I20260811 00:31:21.997764 22694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:21.999733 22694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:22.001042 22694 hybrid_clock.cc:648] HybridClock initialized: now 1786408282000995 us; error 86 us; skew 500 ppm
I20260811 00:31:22.001518 22694 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:31:22.004433 22694 webserver.cc:460] Webserver started at http://127.2.120.62:34775/ using document root <none> and password file <none>
I20260811 00:31:22.005345 22694 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:22.005522 22694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:22.005904 22694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:22.009570 22694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "b3073590d93447c3b2765c6f135834b2"
format_stamp: "Formatted at 2026-08-11 00:31:22 on dist-test-slave-fd7l"
server_key: "2404e0f59acc9ab68e8f0066abbd33ee"
server_key_iv: "a48759634c75d9109a06a076b3d85ef3"
server_key_version: "encryptionkey@0"
I20260811 00:31:22.010492 22694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "b3073590d93447c3b2765c6f135834b2"
format_stamp: "Formatted at 2026-08-11 00:31:22 on dist-test-slave-fd7l"
server_key: "2404e0f59acc9ab68e8f0066abbd33ee"
server_key_iv: "a48759634c75d9109a06a076b3d85ef3"
server_key_version: "encryptionkey@0"
I20260811 00:31:22.016723 22694 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 00:31:22.021209 22713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:22.022907 22694 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:31:22.023175 22694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "b3073590d93447c3b2765c6f135834b2"
format_stamp: "Formatted at 2026-08-11 00:31:22 on dist-test-slave-fd7l"
server_key: "2404e0f59acc9ab68e8f0066abbd33ee"
server_key_iv: "a48759634c75d9109a06a076b3d85ef3"
server_key_version: "encryptionkey@0"
I20260811 00:31:22.023456 22694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:22.041303 22694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:22.042347 22694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:22.042701 22694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:22.065372 22694 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:40369
I20260811 00:31:22.065390 22770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:40369 every 8 connection(s)
I20260811 00:31:22.067004 22694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:31:22.070479 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 22694
I20260811 00:31:22.070690 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:31:22.071272 2528 external_mini_cluster.cc:1428] Setting key 0e2ecadfb0e6b09ca4a52a4c819719c4
I20260811 00:31:22.071892 22771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:22.087990 22771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2: Bootstrap starting.
I20260811 00:31:22.089362 22774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:22.073533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37318 (local address 127.2.120.62:40369)
0811 00:31:22.074684 (+ 1151us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:22.074698 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:22.074744 (+ 46us) server_negotiation.cc:407] Connection header received
0811 00:31:22.077288 (+ 2544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:22.077319 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:22.077989 (+ 670us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:22.078406 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:22.079211 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:22.080726 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:22.082769 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:22.083523 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:22.083710 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:22.085338 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:22.085349 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:22.085368 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:22.086025 (+ 657us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:22.086124 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:22.086149 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:22.086250 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:22.086976 (+ 726us) server_negotiation.cc:299] Negotiation successful
0811 00:31:22.087522 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":441,"threads_started":1}
I20260811 00:31:22.092233 22771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:22.093483 22771 log.cc:824] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:22.096920 22771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2: No bootstrap required, opened a new log
I20260811 00:31:22.105072 22771 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3073590d93447c3b2765c6f135834b2" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40369 } }
I20260811 00:31:22.105577 22771 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:22.105727 22771 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3073590d93447c3b2765c6f135834b2, State: Initialized, Role: FOLLOWER
I20260811 00:31:22.106397 22771 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3073590d93447c3b2765c6f135834b2" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40369 } }
I20260811 00:31:22.106700 22771 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:31:22.106868 22771 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:31:22.107081 22771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:22.109740 22771 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3073590d93447c3b2765c6f135834b2" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40369 } }
I20260811 00:31:22.110424 22771 leader_election.cc:302] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3073590d93447c3b2765c6f135834b2; no voters:
I20260811 00:31:22.111150 22771 leader_election.cc:288] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:31:22.111310 22776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:22.112138 22776 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2 [term 1 LEADER]: Becoming Leader. State: Replica: b3073590d93447c3b2765c6f135834b2, State: Running, Role: LEADER
I20260811 00:31:22.112845 22776 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3073590d93447c3b2765c6f135834b2" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40369 } }
I20260811 00:31:22.113754 22771 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:31:22.116050 22778 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3073590d93447c3b2765c6f135834b2. Latest consensus state: current_term: 1 leader_uuid: "b3073590d93447c3b2765c6f135834b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3073590d93447c3b2765c6f135834b2" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40369 } } }
I20260811 00:31:22.116482 22778 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2 [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:22.117550 22777 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3073590d93447c3b2765c6f135834b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3073590d93447c3b2765c6f135834b2" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 40369 } } }
I20260811 00:31:22.117861 22777 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2 [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:22.119583 22783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:31:22.125603 22783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:31:22.136013 22783 catalog_manager.cc:1347] Generated new cluster ID: d58d870a193f4b6f9319988d89470f16
I20260811 00:31:22.136176 22783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:31:22.147442 22783 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:31:22.149024 22783 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:31:22.159848 22783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2: Generated new TSK 0
I20260811 00:31:22.161008 22783 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:31:22.166827 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:40369
--builtin_ntp_servers=127.2.120.20:46659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 00:31:22.572892 22796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:22.573236 22796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:22.573410 22796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:22.581261 22796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:22.581391 22796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:22.581583 22796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:31:22.592139 22796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:46659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:40369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:22.593670 22796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:22.595772 22796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:22.612010 22803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:22.612234 22802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:22.613682 22805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:22.613996 22796 server_base.cc:1034] running on GCE node
I20260811 00:31:22.614820 22796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:22.616812 22796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:22.618108 22796 hybrid_clock.cc:648] HybridClock initialized: now 1786408282617988 us; error 114 us; skew 500 ppm
I20260811 00:31:22.618575 22796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:31:22.622630 22796 webserver.cc:460] Webserver started at http://127.2.120.1:34173/ using document root <none> and password file <none>
I20260811 00:31:22.623915 22796 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:22.624116 22796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:22.624670 22796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:22.629720 22796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "c5f82b72d5854ceca054b2bf668a7bc3"
format_stamp: "Formatted at 2026-08-11 00:31:22 on dist-test-slave-fd7l"
server_key: "59234b2188832b569feebad8f8b9230c"
server_key_iv: "cc1a0ed973c6defb6eb1ab147170dac6"
server_key_version: "encryptionkey@0"
I20260811 00:31:22.631181 22796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "c5f82b72d5854ceca054b2bf668a7bc3"
format_stamp: "Formatted at 2026-08-11 00:31:22 on dist-test-slave-fd7l"
server_key: "59234b2188832b569feebad8f8b9230c"
server_key_iv: "cc1a0ed973c6defb6eb1ab147170dac6"
server_key_version: "encryptionkey@0"
I20260811 00:31:22.640601 22796 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.009s sys 0.002s
I20260811 00:31:22.646404 22812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:22.648692 22796 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.004s
I20260811 00:31:22.648932 22796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "c5f82b72d5854ceca054b2bf668a7bc3"
format_stamp: "Formatted at 2026-08-11 00:31:22 on dist-test-slave-fd7l"
server_key: "59234b2188832b569feebad8f8b9230c"
server_key_iv: "cc1a0ed973c6defb6eb1ab147170dac6"
server_key_version: "encryptionkey@0"
I20260811 00:31:22.649261 22796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:22.690260 22796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:22.691185 22796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:22.691447 22796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:22.692586 22796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:22.694253 22796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:22.694350 22796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:22.694521 22796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:22.694609 22796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:22.737807 22796 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:46619
I20260811 00:31:22.737891 22934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:46619 every 8 connection(s)
I20260811 00:31:22.741302 22796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:31:22.749930 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 22796
I20260811 00:31:22.750373 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:31:22.751212 2528 external_mini_cluster.cc:1428] Setting key 7309610ba2a9017cb5c490f2d2930926
I20260811 00:31:22.759503 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:40369
--builtin_ntp_servers=127.2.120.20:46659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 00:31:22.770850 22938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:22.749414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:33371 (local address 127.2.120.62:40369)
0811 00:31:22.750994 (+ 1580us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:22.751008 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:22.756185 (+ 5177us) server_negotiation.cc:407] Connection header received
0811 00:31:22.760753 (+ 4568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:22.760760 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:22.760971 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:22.761252 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:22.764358 (+ 3106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:22.765396 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:22.767327 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:22.767786 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:22.767957 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:22.769824 (+ 1867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:22.769830 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:22.769833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:22.769861 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:22.769914 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:22.769929 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:22.770050 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:22.770339 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 00:31:22.770467 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1243,"thread_start_us":656,"threads_started":1}
I20260811 00:31:22.772111 22939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:22.751681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:40369 (local address 127.2.120.1:33371)
0811 00:31:22.755477 (+ 3796us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:22.755695 (+ 218us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:22.759606 (+ 3911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:22.762041 (+ 2435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:22.762065 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:22.762390 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:22.763786 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:22.763824 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:22.765642 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:22.765648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:22.767087 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:22.767107 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:22.768111 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:22.768132 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:22.768401 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:22.769384 (+ 983us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:31:22.769387 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:22.769441 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:31:22.769546 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:22.770053 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:22.770068 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:22.770079 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 00:31:22.770324 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 00:31:22.770830 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1705,"thread_start_us":1121,"threads_started":1}
I20260811 00:31:22.774295 22935 heartbeater.cc:347] Connected to a master server at 127.2.120.62:40369
I20260811 00:31:22.774724 22935 heartbeater.cc:464] Registering TS with master...
I20260811 00:31:22.775906 22935 heartbeater.cc:511] Master 127.2.120.62:40369 requested a full tablet report, sending...
I20260811 00:31:22.778595 22731 ts_manager.cc:194] Registered new tserver with Master: c5f82b72d5854ceca054b2bf668a7bc3 (127.2.120.1:46619)
I20260811 00:31:22.780306 22731 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.120.1:33371
W20260811 00:31:23.165237 22941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:23.165658 22941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:23.165845 22941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:23.173949 22941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:23.174145 22941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:23.174345 22941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:31:23.184903 22941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:46659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:40369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:23.186481 22941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:23.188647 22941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:23.204718 22946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:23.204692 22949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:23.204573 22947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:23.206220 22941 server_base.cc:1034] running on GCE node
I20260811 00:31:23.207098 22941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:23.208389 22941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:23.209714 22941 hybrid_clock.cc:648] HybridClock initialized: now 1786408283209633 us; error 78 us; skew 500 ppm
I20260811 00:31:23.210325 22941 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:31:23.213621 22941 webserver.cc:460] Webserver started at http://127.2.120.2:44001/ using document root <none> and password file <none>
I20260811 00:31:23.214588 22941 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:23.214756 22941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:23.215291 22941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:23.221989 22941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "188b0e39d17940a8904f6bb9aee563ff"
format_stamp: "Formatted at 2026-08-11 00:31:23 on dist-test-slave-fd7l"
server_key: "50e4749d63b966c18c9787dfe39d5085"
server_key_iv: "3080be9aa230840092ab6aa1725c123f"
server_key_version: "encryptionkey@0"
I20260811 00:31:23.224047 22941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "188b0e39d17940a8904f6bb9aee563ff"
format_stamp: "Formatted at 2026-08-11 00:31:23 on dist-test-slave-fd7l"
server_key: "50e4749d63b966c18c9787dfe39d5085"
server_key_iv: "3080be9aa230840092ab6aa1725c123f"
server_key_version: "encryptionkey@0"
I20260811 00:31:23.236720 22941 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.009s sys 0.000s
I20260811 00:31:23.246272 22955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:23.248942 22941 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.004s sys 0.001s
I20260811 00:31:23.249217 22941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "188b0e39d17940a8904f6bb9aee563ff"
format_stamp: "Formatted at 2026-08-11 00:31:23 on dist-test-slave-fd7l"
server_key: "50e4749d63b966c18c9787dfe39d5085"
server_key_iv: "3080be9aa230840092ab6aa1725c123f"
server_key_version: "encryptionkey@0"
I20260811 00:31:23.249486 22941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:23.275048 22941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:23.276156 22941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:23.276559 22941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:23.277733 22941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:23.279462 22941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:23.279558 22941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:23.279739 22941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:23.279839 22941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:23.319978 22941 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:43559
I20260811 00:31:23.320142 23085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:43559 every 8 connection(s)
I20260811 00:31:23.321513 22941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:31:23.327180 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 22941
I20260811 00:31:23.327396 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:31:23.327914 2528 external_mini_cluster.cc:1428] Setting key 7ace5eb749934ceba6bdadf5c9b77aaf
I20260811 00:31:23.331048 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:40369
--builtin_ntp_servers=127.2.120.20:46659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 00:31:23.336196 23089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:23.324846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:44947 (local address 127.2.120.62:40369)
0811 00:31:23.325411 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:23.325418 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:23.326626 (+ 1208us) server_negotiation.cc:407] Connection header received
0811 00:31:23.328956 (+ 2330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:23.328961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:23.329048 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:23.329195 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:23.330568 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.331427 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.333147 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.333571 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.333725 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:23.335266 (+ 1541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:23.335271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:23.335274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:23.335296 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:23.335341 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:23.335354 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:23.335464 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:23.335708 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 00:31:23.335817 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":271,"threads_started":1}
I20260811 00:31:23.337340 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:23.325400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:40369 (local address 127.2.120.2:44947)
0811 00:31:23.326424 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:23.326470 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:23.328614 (+ 2144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:23.329474 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:23.329483 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:23.329611 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:23.330382 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:23.330409 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.331615 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.331619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:23.332981 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:23.332995 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.333825 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.333832 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:23.334081 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:23.334889 (+ 808us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:31:23.334892 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:23.334938 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:31:23.335038 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:23.335462 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:23.335475 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:23.335485 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 00:31:23.335698 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 00:31:23.336139 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":317,"threads_started":1}
I20260811 00:31:23.339084 23086 heartbeater.cc:347] Connected to a master server at 127.2.120.62:40369
I20260811 00:31:23.339445 23086 heartbeater.cc:464] Registering TS with master...
I20260811 00:31:23.340276 23086 heartbeater.cc:511] Master 127.2.120.62:40369 requested a full tablet report, sending...
I20260811 00:31:23.342090 22731 ts_manager.cc:194] Registered new tserver with Master: 188b0e39d17940a8904f6bb9aee563ff (127.2.120.2:43559)
I20260811 00:31:23.343570 22731 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.120.2:44947
W20260811 00:31:23.730605 23091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:23.730938 23091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:23.731034 23091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:23.738945 23091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:23.739071 23091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:23.739259 23091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:31:23.750128 23091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:46659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:40369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:23.751637 23091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:23.753888 23091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:23.770219 23097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:23.770344 23100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:23.770319 23098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:23.771334 23091 server_base.cc:1034] running on GCE node
I20260811 00:31:23.772187 23091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:23.773463 23091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:23.774791 23091 hybrid_clock.cc:648] HybridClock initialized: now 1786408283774706 us; error 70 us; skew 500 ppm
I20260811 00:31:23.775213 23091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:31:23.777886 23091 webserver.cc:460] Webserver started at http://127.2.120.3:41525/ using document root <none> and password file <none>
I20260811 00:31:23.778672 23091 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:23.778823 23091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:23.779191 23091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:23.782488 23091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "dd82731e20664075a8f1d68fc7a76ff0"
format_stamp: "Formatted at 2026-08-11 00:31:23 on dist-test-slave-fd7l"
server_key: "8f980b1346fd43c1957a9e1eb55d7178"
server_key_iv: "ad16d40d30508c39835850b411795b2a"
server_key_version: "encryptionkey@0"
I20260811 00:31:23.783440 23091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "dd82731e20664075a8f1d68fc7a76ff0"
format_stamp: "Formatted at 2026-08-11 00:31:23 on dist-test-slave-fd7l"
server_key: "8f980b1346fd43c1957a9e1eb55d7178"
server_key_iv: "ad16d40d30508c39835850b411795b2a"
server_key_version: "encryptionkey@0"
I20260811 00:31:23.786029 22935 heartbeater.cc:503] Master 127.2.120.62:40369 was elected leader, sending a full tablet report...
I20260811 00:31:23.790198 23091 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 00:31:23.794588 23106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:23.796363 23091 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260811 00:31:23.796975 23091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "dd82731e20664075a8f1d68fc7a76ff0"
format_stamp: "Formatted at 2026-08-11 00:31:23 on dist-test-slave-fd7l"
server_key: "8f980b1346fd43c1957a9e1eb55d7178"
server_key_iv: "ad16d40d30508c39835850b411795b2a"
server_key_version: "encryptionkey@0"
I20260811 00:31:23.797595 23091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:23.843694 23091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:23.844816 23091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:23.845152 23091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:23.846367 23091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:23.848186 23091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:23.848345 23091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:23.848564 23091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:23.848680 23091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 00:31:23.884940 23091 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:33721
I20260811 00:31:23.884976 23235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:33721 every 8 connection(s)
I20260811 00:31:23.886499 23091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:31:23.888311 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 23091
I20260811 00:31:23.888543 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:31:23.889165 2528 external_mini_cluster.cc:1428] Setting key a5b221396cd769ebbf50b4349f775b52
I20260811 00:31:23.901839 23238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:23.889719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:49201 (local address 127.2.120.62:40369)
0811 00:31:23.890273 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:23.890280 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:23.891765 (+ 1485us) server_negotiation.cc:407] Connection header received
0811 00:31:23.894484 (+ 2719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:23.894489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:23.894567 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:23.894681 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:23.895865 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.896798 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.898662 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.899122 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.899231 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:23.900911 (+ 1680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:23.900924 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:23.900927 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:23.900951 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:23.900996 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:23.901009 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:23.901101 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:23.901354 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 00:31:23.901447 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":282,"threads_started":1}
I20260811 00:31:23.903141 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:23.890383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:40369 (local address 127.2.120.3:49201)
0811 00:31:23.891588 (+ 1205us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:23.891642 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:23.894144 (+ 2502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:23.894912 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:23.894921 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:23.895020 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:23.895664 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:23.895689 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.896983 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.896987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:23.898490 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:23.898505 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.899399 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.899406 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:23.899646 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:23.900416 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:31:23.900419 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:23.900476 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:31:23.900675 (+ 199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:23.901124 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:23.901132 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:23.901139 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:31:23.901319 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 00:31:23.901667 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":434,"threads_started":1}
I20260811 00:31:23.904841 23236 heartbeater.cc:347] Connected to a master server at 127.2.120.62:40369
I20260811 00:31:23.905242 23236 heartbeater.cc:464] Registering TS with master...
I20260811 00:31:23.906149 23236 heartbeater.cc:511] Master 127.2.120.62:40369 requested a full tablet report, sending...
I20260811 00:31:23.907745 22731 ts_manager.cc:194] Registered new tserver with Master: dd82731e20664075a8f1d68fc7a76ff0 (127.2.120.3:33721)
I20260811 00:31:23.908772 22731 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.120.3:49201
I20260811 00:31:23.918448 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:31:23.929955 23238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:23.923672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59136 (local address 127.2.120.62:40369)
0811 00:31:23.923876 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:23.923881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:23.924220 (+ 339us) server_negotiation.cc:407] Connection header received
0811 00:31:23.924380 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:23.924384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:23.924463 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:23.924653 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:23.925230 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.925960 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.927413 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.927904 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.928069 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:23.929264 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:23.929269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:23.929272 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:23.929295 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:23.929330 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:23.929342 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:23.929418 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:23.929617 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 00:31:23.929705 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 00:31:23.934386 22731 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:31:23.941689 22731 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:31:23.968693 23255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:23.957206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40250 (local address 127.2.120.1:46619)
0811 00:31:23.958128 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:23.958142 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:23.959571 (+ 1429us) server_negotiation.cc:407] Connection header received
0811 00:31:23.959909 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:23.959922 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:23.960248 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:23.960479 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:23.961229 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.962954 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.965634 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.967188 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.967295 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:23.967549 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:23.968088 (+ 539us) server_negotiation.cc:299] Negotiation successful
0811 00:31:23.968281 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":466,"threads_started":1}
I20260811 00:31:23.968725 23253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:23.956871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:46619 (local address 127.0.0.1:40250)
0811 00:31:23.959391 (+ 2520us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:23.959437 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:23.959641 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:23.960502 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:23.960540 (+ 38us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:23.960574 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:23.960981 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:23.960994 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.963136 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.963140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:23.965372 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:23.965388 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.967697 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.967701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:23.967840 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:23.967873 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 00:31:23.968034 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 00:31:23.968260 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2167,"mutex_wait_us":37,"thread_start_us":332,"threads_started":1}
I20260811 00:31:23.970175 23256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:23.958187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:33721 (local address 127.0.0.1:44058)
0811 00:31:23.960360 (+ 2173us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:23.960383 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:23.960539 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:23.961784 (+ 1245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:23.961788 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:23.961817 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:23.962361 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:23.962375 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.964639 (+ 2264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.964645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:23.966667 (+ 2022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:23.966681 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.968747 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.968756 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:23.969443 (+ 687us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:23.969493 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 00:31:23.969575 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 00:31:23.969740 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2018,"mutex_wait_us":4,"thread_start_us":1226,"threads_started":1}
I20260811 00:31:23.970939 23257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:23.958098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41372 (local address 127.2.120.2:43559)
0811 00:31:23.958731 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:23.958742 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:23.959650 (+ 908us) server_negotiation.cc:407] Connection header received
0811 00:31:23.960846 (+ 1196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:23.960857 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:23.961108 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:23.961290 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:23.962286 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.963943 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.967281 (+ 3338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.968996 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.969161 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:23.969444 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:23.970298 (+ 854us) server_negotiation.cc:299] Negotiation successful
0811 00:31:23.970524 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":301,"threads_started":1}
I20260811 00:31:23.970973 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:23.957533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:43559 (local address 127.0.0.1:41372)
0811 00:31:23.959390 (+ 1857us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:23.959444 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:23.960663 (+ 1219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:23.961580 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:23.961585 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:23.961612 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:23.962088 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:23.962102 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.964107 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.964113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:23.967057 (+ 2944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:23.967074 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.969637 (+ 2563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.969646 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:23.969835 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:23.969870 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 00:31:23.969980 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 00:31:23.970141 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1610,"mutex_wait_us":63,"thread_start_us":2493,"threads_started":1}
I20260811 00:31:23.972241 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:23.959054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44058 (local address 127.2.120.3:33721)
0811 00:31:23.961211 (+ 2157us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:23.961221 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:23.961256 (+ 35us) server_negotiation.cc:407] Connection header received
0811 00:31:23.961349 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:23.961356 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:23.961562 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:23.962612 (+ 1050us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:23.962934 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.964408 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.966878 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:23.968293 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:23.971401 (+ 3108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:23.971636 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:23.971704 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 00:31:23.971839 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1972,"thread_start_us":2850,"threads_started":1}
I20260811 00:31:23.973294 22866 tablet_service.cc:1467] Processing CreateTablet for tablet 2357ec8ff78941578b62510d8c25fb74 (DEFAULT_TABLE table=test-table [id=493e422b202d42f0845e24e626445175]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:31:23.975131 22866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2357ec8ff78941578b62510d8c25fb74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:23.975008 23013 tablet_service.cc:1467] Processing CreateTablet for tablet 2357ec8ff78941578b62510d8c25fb74 (DEFAULT_TABLE table=test-table [id=493e422b202d42f0845e24e626445175]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:31:23.976362 23013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2357ec8ff78941578b62510d8c25fb74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:23.976187 23164 tablet_service.cc:1467] Processing CreateTablet for tablet 2357ec8ff78941578b62510d8c25fb74 (DEFAULT_TABLE table=test-table [id=493e422b202d42f0845e24e626445175]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:31:23.977733 23164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2357ec8ff78941578b62510d8c25fb74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:23.989728 23260 tablet_bootstrap.cc:492] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff: Bootstrap starting.
I20260811 00:31:23.994499 23260 tablet_bootstrap.cc:654] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:23.995805 23261 tablet_bootstrap.cc:492] T 2357ec8ff78941578b62510d8c25fb74 P c5f82b72d5854ceca054b2bf668a7bc3: Bootstrap starting.
I20260811 00:31:23.996186 23260 log.cc:824] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:24.001681 23261 tablet_bootstrap.cc:654] T 2357ec8ff78941578b62510d8c25fb74 P c5f82b72d5854ceca054b2bf668a7bc3: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:24.003954 23262 tablet_bootstrap.cc:492] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0: Bootstrap starting.
I20260811 00:31:24.004134 23261 log.cc:824] T 2357ec8ff78941578b62510d8c25fb74 P c5f82b72d5854ceca054b2bf668a7bc3: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:24.009454 23262 tablet_bootstrap.cc:654] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:24.010394 23260 tablet_bootstrap.cc:492] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff: No bootstrap required, opened a new log
I20260811 00:31:24.010411 23261 tablet_bootstrap.cc:492] T 2357ec8ff78941578b62510d8c25fb74 P c5f82b72d5854ceca054b2bf668a7bc3: No bootstrap required, opened a new log
I20260811 00:31:24.010761 23260 ts_tablet_manager.cc:1397] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff: Time spent bootstrapping tablet: real 0.021s user 0.013s sys 0.004s
I20260811 00:31:24.010829 23261 ts_tablet_manager.cc:1397] T 2357ec8ff78941578b62510d8c25fb74 P c5f82b72d5854ceca054b2bf668a7bc3: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.004s
I20260811 00:31:24.010988 23262 log.cc:824] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:24.015144 23262 tablet_bootstrap.cc:492] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0: No bootstrap required, opened a new log
I20260811 00:31:24.015601 23262 ts_tablet_manager.cc:1397] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.006s
I20260811 00:31:24.022401 23260 raft_consensus.cc:348] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f82b72d5854ceca054b2bf668a7bc3" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 46619 } } peers { permanent_uuid: "dd82731e20664075a8f1d68fc7a76ff0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 33721 } } peers { permanent_uuid: "188b0e39d17940a8904f6bb9aee563ff" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43559 } }
I20260811 00:31:24.022925 23260 raft_consensus.cc:374] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:24.022656 23261 raft_consensus.cc:348] T 2357ec8ff78941578b62510d8c25fb74 P c5f82b72d5854ceca054b2bf668a7bc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f82b72d5854ceca054b2bf668a7bc3" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 46619 } } peers { permanent_uuid: "dd82731e20664075a8f1d68fc7a76ff0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 33721 } } peers { permanent_uuid: "188b0e39d17940a8904f6bb9aee563ff" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43559 } }
I20260811 00:31:24.023030 23260 raft_consensus.cc:729] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 188b0e39d17940a8904f6bb9aee563ff, State: Initialized, Role: FOLLOWER
I20260811 00:31:24.023182 23261 raft_consensus.cc:374] T 2357ec8ff78941578b62510d8c25fb74 P c5f82b72d5854ceca054b2bf668a7bc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:24.023293 23261 raft_consensus.cc:729] T 2357ec8ff78941578b62510d8c25fb74 P c5f82b72d5854ceca054b2bf668a7bc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5f82b72d5854ceca054b2bf668a7bc3, State: Initialized, Role: FOLLOWER
I20260811 00:31:24.023826 23260 consensus_queue.cc:260] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f82b72d5854ceca054b2bf668a7bc3" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 46619 } } peers { permanent_uuid: "dd82731e20664075a8f1d68fc7a76ff0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 33721 } } peers { permanent_uuid: "188b0e39d17940a8904f6bb9aee563ff" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43559 } }
I20260811 00:31:24.023948 23261 consensus_queue.cc:260] T 2357ec8ff78941578b62510d8c25fb74 P c5f82b72d5854ceca054b2bf668a7bc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f82b72d5854ceca054b2bf668a7bc3" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 46619 } } peers { permanent_uuid: "dd82731e20664075a8f1d68fc7a76ff0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 33721 } } peers { permanent_uuid: "188b0e39d17940a8904f6bb9aee563ff" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43559 } }
I20260811 00:31:24.025200 23086 heartbeater.cc:503] Master 127.2.120.62:40369 was elected leader, sending a full tablet report...
I20260811 00:31:24.026038 23260 ts_tablet_manager.cc:1428] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff: Time spent starting tablet: real 0.015s user 0.012s sys 0.003s
I20260811 00:31:24.028216 23261 ts_tablet_manager.cc:1428] T 2357ec8ff78941578b62510d8c25fb74 P c5f82b72d5854ceca054b2bf668a7bc3: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
I20260811 00:31:24.028846 23262 raft_consensus.cc:348] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f82b72d5854ceca054b2bf668a7bc3" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 46619 } } peers { permanent_uuid: "dd82731e20664075a8f1d68fc7a76ff0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 33721 } } peers { permanent_uuid: "188b0e39d17940a8904f6bb9aee563ff" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43559 } }
I20260811 00:31:24.029260 23262 raft_consensus.cc:374] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:24.029338 23262 raft_consensus.cc:729] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd82731e20664075a8f1d68fc7a76ff0, State: Initialized, Role: FOLLOWER
I20260811 00:31:24.030023 23262 consensus_queue.cc:260] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f82b72d5854ceca054b2bf668a7bc3" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 46619 } } peers { permanent_uuid: "dd82731e20664075a8f1d68fc7a76ff0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 33721 } } peers { permanent_uuid: "188b0e39d17940a8904f6bb9aee563ff" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43559 } }
I20260811 00:31:24.030527 23267 raft_consensus.cc:482] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:31:24.030889 23267 raft_consensus.cc:504] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f82b72d5854ceca054b2bf668a7bc3" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 46619 } } peers { permanent_uuid: "dd82731e20664075a8f1d68fc7a76ff0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 33721 } } peers { permanent_uuid: "188b0e39d17940a8904f6bb9aee563ff" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43559 } }
I20260811 00:31:24.033895 23267 leader_election.cc:288] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5f82b72d5854ceca054b2bf668a7bc3 (127.2.120.1:46619), dd82731e20664075a8f1d68fc7a76ff0 (127.2.120.3:33721)
I20260811 00:31:24.042337 23236 heartbeater.cc:503] Master 127.2.120.62:40369 was elected leader, sending a full tablet report...
I20260811 00:31:24.042986 23262 ts_tablet_manager.cc:1428] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0: Time spent starting tablet: real 0.027s user 0.019s sys 0.007s
I20260811 00:31:24.047050 23270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:24.034319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:46619 (local address 127.2.120.2:47233)
0811 00:31:24.037234 (+ 2915us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:24.037264 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:24.037582 (+ 318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:24.038295 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:24.038303 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:24.038343 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:24.038943 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:24.038966 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:24.041001 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:24.041010 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:24.043690 (+ 2680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:24.043712 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:24.045763 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:24.045769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:24.046312 (+ 543us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:24.046384 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 00:31:24.046529 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 00:31:24.046737 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2687,"mutex_wait_us":32,"thread_start_us":2447,"threads_started":1}
I20260811 00:31:24.047991 23255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:24.034888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:47233 (local address 127.2.120.1:46619)
0811 00:31:24.035111 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:24.035120 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:24.037409 (+ 2289us) server_negotiation.cc:407] Connection header received
0811 00:31:24.037796 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:24.037806 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:24.037930 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:24.038028 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:24.039181 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:24.040724 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:24.043983 (+ 3259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:24.045311 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:24.045460 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:24.045579 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:24.047428 (+ 1849us) server_negotiation.cc:299] Negotiation successful
0811 00:31:24.047585 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:31:24.049784 22887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2357ec8ff78941578b62510d8c25fb74" candidate_uuid: "188b0e39d17940a8904f6bb9aee563ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5f82b72d5854ceca054b2bf668a7bc3" is_pre_election: true
I20260811 00:31:24.050494 22887 raft_consensus.cc:2454] T 2357ec8ff78941578b62510d8c25fb74 P c5f82b72d5854ceca054b2bf668a7bc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 188b0e39d17940a8904f6bb9aee563ff in term 0.
I20260811 00:31:24.051816 22958 leader_election.cc:302] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 188b0e39d17940a8904f6bb9aee563ff, c5f82b72d5854ceca054b2bf668a7bc3; no voters:
I20260811 00:31:24.052587 23267 raft_consensus.cc:2781] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:31:24.053077 23267 raft_consensus.cc:482] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:31:24.053258 23267 raft_consensus.cc:3037] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:24.053746 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:24.037609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:49499 (local address 127.2.120.3:33721)
0811 00:31:24.038301 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:24.038312 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:24.045309 (+ 6997us) server_negotiation.cc:407] Connection header received
0811 00:31:24.045481 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:24.045489 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:24.045582 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:24.045708 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:24.046715 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:24.047707 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:24.051139 (+ 3432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:24.052484 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:24.052731 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:24.052852 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:24.053304 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 00:31:24.053452 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500}
I20260811 00:31:24.053870 23271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:24.037819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:33721 (local address 127.2.120.2:49499)
0811 00:31:24.045133 (+ 7314us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:24.045159 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:24.045337 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:24.045754 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:24.045759 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:24.045789 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:24.046506 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:24.046525 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:24.048113 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:24.048126 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:24.050667 (+ 2541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:24.050692 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:24.052806 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:24.052815 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:24.053112 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:24.053142 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 00:31:24.053274 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 00:31:24.053474 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7139,"mutex_wait_us":1,"thread_start_us":6923,"threads_started":1}
I20260811 00:31:24.055168 23189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2357ec8ff78941578b62510d8c25fb74" candidate_uuid: "188b0e39d17940a8904f6bb9aee563ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd82731e20664075a8f1d68fc7a76ff0" is_pre_election: true
I20260811 00:31:24.055837 23189 raft_consensus.cc:2454] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 188b0e39d17940a8904f6bb9aee563ff in term 0.
I20260811 00:31:24.057137 23267 raft_consensus.cc:504] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f82b72d5854ceca054b2bf668a7bc3" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 46619 } } peers { permanent_uuid: "dd82731e20664075a8f1d68fc7a76ff0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 33721 } } peers { permanent_uuid: "188b0e39d17940a8904f6bb9aee563ff" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43559 } }
I20260811 00:31:24.057750 23267 leader_election.cc:288] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [CANDIDATE]: Term 1 election: Requested vote from peers c5f82b72d5854ceca054b2bf668a7bc3 (127.2.120.1:46619), dd82731e20664075a8f1d68fc7a76ff0 (127.2.120.3:33721)
I20260811 00:31:24.058498 22887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2357ec8ff78941578b62510d8c25fb74" candidate_uuid: "188b0e39d17940a8904f6bb9aee563ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5f82b72d5854ceca054b2bf668a7bc3"
I20260811 00:31:24.058552 23189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2357ec8ff78941578b62510d8c25fb74" candidate_uuid: "188b0e39d17940a8904f6bb9aee563ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd82731e20664075a8f1d68fc7a76ff0"
I20260811 00:31:24.058758 22887 raft_consensus.cc:3037] T 2357ec8ff78941578b62510d8c25fb74 P c5f82b72d5854ceca054b2bf668a7bc3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:24.058876 23189 raft_consensus.cc:3037] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:24.061403 22887 raft_consensus.cc:2454] T 2357ec8ff78941578b62510d8c25fb74 P c5f82b72d5854ceca054b2bf668a7bc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 188b0e39d17940a8904f6bb9aee563ff in term 1.
I20260811 00:31:24.061897 23189 raft_consensus.cc:2454] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 188b0e39d17940a8904f6bb9aee563ff in term 1.
I20260811 00:31:24.061975 22958 leader_election.cc:302] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 188b0e39d17940a8904f6bb9aee563ff, c5f82b72d5854ceca054b2bf668a7bc3; no voters:
I20260811 00:31:24.062422 23267 raft_consensus.cc:2781] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:24.063028 23267 raft_consensus.cc:686] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [term 1 LEADER]: Becoming Leader. State: Replica: 188b0e39d17940a8904f6bb9aee563ff, State: Running, Role: LEADER
I20260811 00:31:24.063644 23267 consensus_queue.cc:237] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f82b72d5854ceca054b2bf668a7bc3" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 46619 } } peers { permanent_uuid: "dd82731e20664075a8f1d68fc7a76ff0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 33721 } } peers { permanent_uuid: "188b0e39d17940a8904f6bb9aee563ff" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43559 } }
I20260811 00:31:24.071686 22731 catalog_manager.cc:5495] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 188b0e39d17940a8904f6bb9aee563ff (127.2.120.2). New cstate: current_term: 1 leader_uuid: "188b0e39d17940a8904f6bb9aee563ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f82b72d5854ceca054b2bf668a7bc3" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 46619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd82731e20664075a8f1d68fc7a76ff0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 33721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "188b0e39d17940a8904f6bb9aee563ff" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 43559 } health_report { overall_health: HEALTHY } } }
W20260811 00:31:24.074100 23088 tablet.cc:2369] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:31:24.116206 23257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:24.108456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41374 (local address 127.2.120.2:43559)
0811 00:31:24.108767 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:24.108773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:24.108799 (+ 26us) server_negotiation.cc:407] Connection header received
0811 00:31:24.108950 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:24.108955 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:24.109099 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:24.109319 (+ 220us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:31:24.109989 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:24.110925 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:24.112875 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:24.113388 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:24.113524 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:24.114720 (+ 1196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:31:24.115369 (+ 649us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:31:24.115516 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:24.115825 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 00:31:24.115934 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260811 00:31:24.125913 22887 raft_consensus.cc:1261] T 2357ec8ff78941578b62510d8c25fb74 P c5f82b72d5854ceca054b2bf668a7bc3 [term 1 FOLLOWER]: Refusing update from remote peer 188b0e39d17940a8904f6bb9aee563ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:31:24.125913 23189 raft_consensus.cc:1261] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0 [term 1 FOLLOWER]: Refusing update from remote peer 188b0e39d17940a8904f6bb9aee563ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:31:24.127076 23273 consensus_queue.cc:1035] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd82731e20664075a8f1d68fc7a76ff0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:31:24.127497 23267 consensus_queue.cc:1035] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5f82b72d5854ceca054b2bf668a7bc3" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 46619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 00:31:24.138435 23237 tablet.cc:2369] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:31:24.142033 23277 mvcc.cc:205] Tried to move back new op lower bound from 7317128331767717888 to 7317128331530346496. Current Snapshot: MvccSnapshot[applied={T|T < 7317128331767717888}]
I20260811 00:31:24.142680 23279 mvcc.cc:205] Tried to move back new op lower bound from 7317128331767717888 to 7317128331530346496. Current Snapshot: MvccSnapshot[applied={T|T < 7317128331767717888}]
I20260811 00:31:24.161576 22731 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:31:24.161852 22731 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:31:24.166518 22731 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b3073590d93447c3b2765c6f135834b2: Sending DeleteTablet for 3 replicas of tablet 2357ec8ff78941578b62510d8c25fb74
I20260811 00:31:24.168023 23013 tablet_service.cc:1514] Processing DeleteTablet for tablet 2357ec8ff78941578b62510d8c25fb74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:31:24 UTC) from {username='slave'} at 127.0.0.1:41372
I20260811 00:31:24.168016 23164 tablet_service.cc:1514] Processing DeleteTablet for tablet 2357ec8ff78941578b62510d8c25fb74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:31:24 UTC) from {username='slave'} at 127.0.0.1:44058
I20260811 00:31:24.168895 22866 tablet_service.cc:1514] Processing DeleteTablet for tablet 2357ec8ff78941578b62510d8c25fb74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:31:24 UTC) from {username='slave'} at 127.0.0.1:40250
I20260811 00:31:24.169582 23287 tablet_replica.cc:330] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0: stopping tablet replica
I20260811 00:31:24.169749 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 22796
I20260811 00:31:24.170719 23287 raft_consensus.cc:2229] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:31:24.172073 23287 raft_consensus.cc:2258] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:31:24.179265 23289 tablet_replica.cc:330] T 2357ec8ff78941578b62510d8c25fb74 P c5f82b72d5854ceca054b2bf668a7bc3: stopping tablet replica
I20260811 00:31:24.180790 23289 raft_consensus.cc:2229] T 2357ec8ff78941578b62510d8c25fb74 P c5f82b72d5854ceca054b2bf668a7bc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:31:24.182662 23287 ts_tablet_manager.cc:1905] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:31:24.182662 23289 raft_consensus.cc:2258] T 2357ec8ff78941578b62510d8c25fb74 P c5f82b72d5854ceca054b2bf668a7bc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:31:24.185109 23288 tablet_replica.cc:330] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff: stopping tablet replica
I20260811 00:31:24.187650 23288 raft_consensus.cc:2229] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:31:24.191860 23289 ts_tablet_manager.cc:1905] T 2357ec8ff78941578b62510d8c25fb74 P c5f82b72d5854ceca054b2bf668a7bc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:31:24.193722 23288 raft_consensus.cc:2258] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:31:24.198747 23287 ts_tablet_manager.cc:1918] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:31:24.199216 23287 log.cc:1196] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/2357ec8ff78941578b62510d8c25fb74
I20260811 00:31:24.200318 23287 ts_tablet_manager.cc:1939] T 2357ec8ff78941578b62510d8c25fb74 P dd82731e20664075a8f1d68fc7a76ff0: Deleting consensus metadata
I20260811 00:31:24.198933 23288 ts_tablet_manager.cc:1905] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:31:24.204366 23255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:24.178427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40256 (local address 127.2.120.1:46619)
0811 00:31:24.179040 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:24.179055 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:24.188252 (+ 9197us) server_negotiation.cc:407] Connection header received
0811 00:31:24.189298 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:24.189310 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:24.189499 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:24.189620 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:24.190787 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:24.192853 (+ 2066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:24.196963 (+ 4110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:24.197886 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:24.198150 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:24.201193 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:24.201215 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:24.201249 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:24.202653 (+ 1404us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:24.202768 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:24.202808 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:24.203011 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:24.203570 (+ 559us) server_negotiation.cc:299] Negotiation successful
0811 00:31:24.203786 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244}
I20260811 00:31:24.204399 22716 catalog_manager.cc:4841] TS dd82731e20664075a8f1d68fc7a76ff0 (127.2.120.3:33721): tablet 2357ec8ff78941578b62510d8c25fb74 (table test-table [id=493e422b202d42f0845e24e626445175]) successfully deleted
I20260811 00:31:24.206727 22930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:24.218189 23288 ts_tablet_manager.cc:1918] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:31:24.218557 23288 log.cc:1196] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/2357ec8ff78941578b62510d8c25fb74
I20260811 00:31:24.219719 23288 ts_tablet_manager.cc:1939] T 2357ec8ff78941578b62510d8c25fb74 P 188b0e39d17940a8904f6bb9aee563ff: Deleting consensus metadata
I20260811 00:31:24.223461 22717 catalog_manager.cc:4841] TS 188b0e39d17940a8904f6bb9aee563ff (127.2.120.2:43559): tablet 2357ec8ff78941578b62510d8c25fb74 (table test-table [id=493e422b202d42f0845e24e626445175]) successfully deleted
I20260811 00:31:24.312981 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 22796
W20260811 00:31:24.337756 22716 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.120.1:46619 (error 108)
I20260811 00:31:24.338702 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 22941
I20260811 00:31:24.352977 23257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:24.340918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41378 (local address 127.2.120.2:43559)
0811 00:31:24.341165 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:24.341175 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:24.341203 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:31:24.341410 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:24.341417 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:24.341585 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:24.341714 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:24.342647 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:24.344338 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:24.347423 (+ 3085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:24.348476 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:24.348697 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:24.350639 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:24.350648 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:24.350663 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:24.351327 (+ 664us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:24.351386 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:24.351408 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:24.351599 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:24.352282 (+ 683us) server_negotiation.cc:299] Negotiation successful
0811 00:31:24.352425 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 00:31:24.353230 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:24.352682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:46619 (local address 127.0.0.1:40260)
0811 00:31:24.352907 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:24.353049 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 00:31:24.354178 23080 generic_service.cc:196] Checking for leaks (request via RPC)
W20260811 00:31:24.354120 22716 catalog_manager.cc:4584] TS c5f82b72d5854ceca054b2bf668a7bc3 (127.2.120.1:46619): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2357ec8ff78941578b62510d8c25fb74: Network error: Client connection negotiation failed: client connection to 127.2.120.1:46619: connect: Connection refused (error 111)
I20260811 00:31:24.408263 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:24.407696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:46619 (local address 127.0.0.1:40268)
0811 00:31:24.407972 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:24.408068 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260811 00:31:24.409720 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:24.409211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:46619 (local address 127.0.0.1:40280)
0811 00:31:24.409430 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:24.409546 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 00:31:24.470772 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:24.470143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:46619 (local address 127.0.0.1:40286)
0811 00:31:24.470399 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:24.470500 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260811 00:31:24.472285 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:24.471652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:46619 (local address 127.0.0.1:40298)
0811 00:31:24.471937 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:24.472042 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260811 00:31:24.480880 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 22941
I20260811 00:31:24.507637 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 23091
I20260811 00:31:24.515239 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:24.508445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44068 (local address 127.2.120.3:33721)
0811 00:31:24.508682 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:24.508687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:24.508708 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:31:24.508936 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:24.508941 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:24.509027 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:24.509127 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:24.509559 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:24.510369 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:24.511726 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:24.512146 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:24.512254 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:24.513683 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:24.513695 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:24.513712 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:24.514350 (+ 638us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:24.514412 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:24.514434 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:24.514537 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:24.514796 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 00:31:24.514916 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260811 00:31:24.515995 23231 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:24.553432 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:24.552654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:46619 (local address 127.0.0.1:40300)
0811 00:31:24.553021 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:24.553173 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260811 00:31:24.555147 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:24.554532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:46619 (local address 127.0.0.1:40302)
0811 00:31:24.554828 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:24.554970 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260811 00:31:24.612682 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 23091
I20260811 00:31:24.635262 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 22694
I20260811 00:31:24.637034 22764 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:24.753168 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:24.752470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:46619 (local address 127.0.0.1:40312)
0811 00:31:24.752733 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:24.752848 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 00:31:24.753443 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 22694
2026-08-11T00:31:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3292 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:31:24 dist-test-slave-fd7l krb5kdc[23299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:31:24 dist-test-slave-fd7l krb5kdc[23299](info): set up 2 sockets
Aug 11 00:31:24 dist-test-slave-fd7l krb5kdc[23299](info): commencing operation
krb5kdc: starting...
W20260811 00:31:26.048687 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.003s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:31:26 dist-test-slave-fd7l krb5kdc[23299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:31:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:31:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:31:26.426244 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:44433
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:38203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:44433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:31:26.863343 23312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:26.863732 23312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:26.863919 23312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:26.874878 23312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:31:26.875066 23312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:26.875445 23312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:26.875718 23312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:31:26.875806 23312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:31:26.889304 23312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:38203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:44433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:44433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:26.891428 23312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:26.893417 23312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:26.910219 23319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:26.910444 23318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:26.910429 23321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:26.911403 23312 server_base.cc:1034] running on GCE node
I20260811 00:31:26.912930 23312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:26.914949 23312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:26.916309 23312 hybrid_clock.cc:648] HybridClock initialized: now 1786408286916219 us; error 72 us; skew 500 ppm
I20260811 00:31:26.917111 23312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:26 dist-test-slave-fd7l krb5kdc[23299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408286, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:26.927358 23312 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:31:26.929612 23312 webserver.cc:460] Webserver started at http://127.2.120.62:42231/ using document root <none> and password file <none>
I20260811 00:31:26.930562 23312 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:26.930768 23312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:26.931177 23312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:26.935770 23312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "27c1e565ec9d48148d38672c4389e78f"
format_stamp: "Formatted at 2026-08-11 00:31:26 on dist-test-slave-fd7l"
server_key: "ccf09b27638ff8c39d920921249a3f93"
server_key_iv: "acb5752239079ca51e559c88fe4e811d"
server_key_version: "encryptionkey@0"
I20260811 00:31:26.936894 23312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "27c1e565ec9d48148d38672c4389e78f"
format_stamp: "Formatted at 2026-08-11 00:31:26 on dist-test-slave-fd7l"
server_key: "ccf09b27638ff8c39d920921249a3f93"
server_key_iv: "acb5752239079ca51e559c88fe4e811d"
server_key_version: "encryptionkey@0"
I20260811 00:31:26.943790 23312 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260811 00:31:26.949774 23328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:26.951604 23312 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:31:26.951889 23312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "27c1e565ec9d48148d38672c4389e78f"
format_stamp: "Formatted at 2026-08-11 00:31:26 on dist-test-slave-fd7l"
server_key: "ccf09b27638ff8c39d920921249a3f93"
server_key_iv: "acb5752239079ca51e559c88fe4e811d"
server_key_version: "encryptionkey@0"
I20260811 00:31:26.952168 23312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:26.991852 23312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:26.997495 23312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:26.997895 23312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:27.024111 23312 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:44433
I20260811 00:31:27.024135 23389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:44433 every 8 connection(s)
I20260811 00:31:27.025827 23312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:31:27.028391 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 23312
I20260811 00:31:27.028644 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:31:27.029292 2528 external_mini_cluster.cc:1428] Setting key e6dab10d49a5d2e9b7b8230b0eb015b9
I20260811 00:31:27.031355 23391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:27.047735 23391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f: Bootstrap starting.
Aug 11 00:31:27 dist-test-slave-fd7l krb5kdc[23299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:31:27.051738 23391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:27.053521 23391 log.cc:824] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:27.058225 23391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f: No bootstrap required, opened a new log
I20260811 00:31:27.061231 23394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:27.031795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47424 (local address 127.2.120.62:44433)
0811 00:31:27.032782 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:27.032794 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:27.032839 (+ 45us) server_negotiation.cc:407] Connection header received
0811 00:31:27.034587 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:27.034614 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:27.035084 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:27.035495 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:27.037858 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:27.039829 (+ 1971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:27.042197 (+ 2368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:27.043118 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:27.043331 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:27.050303 (+ 6972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:27.050319 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:27.050345 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:27.050418 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:27.055774 (+ 5356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:27.056949 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:27.056957 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:27.056965 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:27.057262 (+ 297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:27.057739 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:27.057749 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:27.057751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:27.058503 (+ 752us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:27.058892 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:27.059435 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 00:31:27.059952 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":451,"threads_started":1}
I20260811 00:31:27.067974 23391 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c1e565ec9d48148d38672c4389e78f" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44433 } }
I20260811 00:31:27.068516 23391 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:27.068717 23391 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27c1e565ec9d48148d38672c4389e78f, State: Initialized, Role: FOLLOWER
I20260811 00:31:27.069589 23391 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c1e565ec9d48148d38672c4389e78f" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44433 } }
I20260811 00:31:27.069955 23391 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:31:27.070137 23391 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:31:27.070380 23391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:27.073200 23391 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c1e565ec9d48148d38672c4389e78f" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44433 } }
I20260811 00:31:27.073971 23391 leader_election.cc:302] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27c1e565ec9d48148d38672c4389e78f; no voters:
I20260811 00:31:27.075063 23391 leader_election.cc:288] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:31:27.075227 23396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:27.076028 23396 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f [term 1 LEADER]: Becoming Leader. State: Replica: 27c1e565ec9d48148d38672c4389e78f, State: Running, Role: LEADER
I20260811 00:31:27.077138 23396 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c1e565ec9d48148d38672c4389e78f" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44433 } }
I20260811 00:31:27.078006 23391 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:31:27.081423 23398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27c1e565ec9d48148d38672c4389e78f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c1e565ec9d48148d38672c4389e78f" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44433 } } }
I20260811 00:31:27.081938 23398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:27.082449 23399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27c1e565ec9d48148d38672c4389e78f. Latest consensus state: current_term: 1 leader_uuid: "27c1e565ec9d48148d38672c4389e78f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c1e565ec9d48148d38672c4389e78f" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 44433 } } }
I20260811 00:31:27.082763 23399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:27.089301 23407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:31:27.095647 23407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:31:27.108135 23407 catalog_manager.cc:1347] Generated new cluster ID: b4e78de61ce74e4b83ba9f7e47715b78
I20260811 00:31:27.108311 23407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:31:27.143502 23407 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:31:27.145669 23407 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:31:27.177903 23407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f: Generated new TSK 0
I20260811 00:31:27.179090 23407 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
I20260811 00:31:27.263839 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:44433
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:38203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:31:27.664041 23422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:27.664410 23422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:27.664592 23422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:27.673281 23422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:27.673439 23422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:27.673573 23422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:31:27.684500 23422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:38203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:44433
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:27.685956 23422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:27.687281 23422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:27.703572 23431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:27.703619 23428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:27.703529 23427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:27.706574 23422 server_base.cc:1034] running on GCE node
I20260811 00:31:27.707304 23422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:27.708374 23422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:27.709614 23422 hybrid_clock.cc:648] HybridClock initialized: now 1786408287709567 us; error 49 us; skew 500 ppm
I20260811 00:31:27.709942 23422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:27 dist-test-slave-fd7l krb5kdc[23299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408287, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:27.713389 23422 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:31:27.715214 23422 webserver.cc:460] Webserver started at http://127.2.120.1:43111/ using document root <none> and password file <none>
I20260811 00:31:27.716001 23422 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:27.716121 23422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:27.716987 23422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:27.720860 23422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "14e3ceb6d95248a9a5ea8598a3faa5b6"
format_stamp: "Formatted at 2026-08-11 00:31:27 on dist-test-slave-fd7l"
server_key: "4f19f19d2dcad80984c4b8015d12e490"
server_key_iv: "656bee53bd05682a382bf8fbf72e0cba"
server_key_version: "encryptionkey@0"
I20260811 00:31:27.721804 23422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "14e3ceb6d95248a9a5ea8598a3faa5b6"
format_stamp: "Formatted at 2026-08-11 00:31:27 on dist-test-slave-fd7l"
server_key: "4f19f19d2dcad80984c4b8015d12e490"
server_key_iv: "656bee53bd05682a382bf8fbf72e0cba"
server_key_version: "encryptionkey@0"
I20260811 00:31:27.728955 23422 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260811 00:31:27.733143 23440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:27.734678 23422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:31:27.734833 23422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "14e3ceb6d95248a9a5ea8598a3faa5b6"
format_stamp: "Formatted at 2026-08-11 00:31:27 on dist-test-slave-fd7l"
server_key: "4f19f19d2dcad80984c4b8015d12e490"
server_key_iv: "656bee53bd05682a382bf8fbf72e0cba"
server_key_version: "encryptionkey@0"
I20260811 00:31:27.735044 23422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:27.755156 23422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:27.760906 23422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:27.761210 23422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:27.762316 23422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:27.763960 23422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:27.764068 23422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:27.764169 23422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:27.764233 23422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:27.809770 23422 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:36125
I20260811 00:31:27.809788 23572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:36125 every 8 connection(s)
I20260811 00:31:27.812299 23422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:31:27.820742 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 23422
I20260811 00:31:27.821046 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:31:27.821724 2528 external_mini_cluster.cc:1428] Setting key 6533dbb707e0f223aeee922b7738ceba
Aug 11 00:31:27 dist-test-slave-fd7l krb5kdc[23299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408287, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:31:27.844836 23575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:27.816443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:47897 (local address 127.2.120.62:44433)
0811 00:31:27.817106 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:27.817111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:27.818761 (+ 1650us) server_negotiation.cc:407] Connection header received
0811 00:31:27.821445 (+ 2684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:27.821452 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:27.821575 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:27.821728 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:27.824364 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:27.825513 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:27.829425 (+ 3912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:27.830028 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:27.830207 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:27.836489 (+ 6282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:27.836497 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:27.836501 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:27.836619 (+ 118us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:27.840023 (+ 3404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:27.841397 (+ 1374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:27.841405 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:27.841407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:27.841625 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:27.842299 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:27.842304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:27.842306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:27.842742 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:27.842956 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:27.843993 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0811 00:31:27.844340 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":302,"threads_started":1}
I20260811 00:31:27.846347 23576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:27.817151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44433 (local address 127.2.120.1:47897)
0811 00:31:27.818396 (+ 1245us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:27.818466 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:27.820911 (+ 2445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:27.822094 (+ 1183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:27.822108 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:27.822874 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:27.823942 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:27.823978 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:27.825846 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:27.825855 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:27.829001 (+ 3146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:27.829046 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:27.830434 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:27.830446 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:27.830979 (+ 533us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:27.832066 (+ 1087us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:31:27.832120 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:27.835995 (+ 3875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:27.840279 (+ 4284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:27.840293 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:27.840323 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:27.841159 (+ 836us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:27.841792 (+ 633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:27.841798 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:27.841804 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:27.842109 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:27.842928 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:27.842941 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:27.843477 (+ 536us) client_negotiation.cc:769] Sending connection context
0811 00:31:27.843844 (+ 367us) client_negotiation.cc:240] Negotiation successful
0811 00:31:27.844646 (+ 802us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"thread_start_us":482,"threads_started":1}
I20260811 00:31:27.850404 23573 heartbeater.cc:347] Connected to a master server at 127.2.120.62:44433
I20260811 00:31:27.851066 23573 heartbeater.cc:464] Registering TS with master...
I20260811 00:31:27.852360 23573 heartbeater.cc:511] Master 127.2.120.62:44433 requested a full tablet report, sending...
I20260811 00:31:27.857128 23351 ts_manager.cc:194] Registered new tserver with Master: 14e3ceb6d95248a9a5ea8598a3faa5b6 (127.2.120.1:36125)
I20260811 00:31:27.860410 23351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:47897
WARNING: no policy specified for kudu/127.2.120.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
W20260811 00:31:27.932392 2528 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.2.120.2: real 0.106s user 0.003s sys 0.032s
I20260811 00:31:27.933904 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:44433
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:38203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:31:28.343341 23579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:28.343710 23579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:28.343904 23579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:28.352703 23579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:28.352861 23579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:28.353078 23579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:31:28.364615 23579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:38203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:44433
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:28.366240 23579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:28.367836 23579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:28.384472 23589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:28.384629 23587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:28.384590 23586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:28.385511 23579 server_base.cc:1034] running on GCE node
I20260811 00:31:28.386286 23579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:28.387496 23579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:28.390950 23579 hybrid_clock.cc:648] HybridClock initialized: now 1786408288390874 us; error 59 us; skew 500 ppm
I20260811 00:31:28.391410 23579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:28 dist-test-slave-fd7l krb5kdc[23299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408288, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:28.395759 23579 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:31:28.397710 23579 webserver.cc:460] Webserver started at http://127.2.120.2:37513/ using document root <none> and password file <none>
I20260811 00:31:28.399111 23579 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:28.399307 23579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:28.399786 23579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:28.403525 23579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "5f6ffc20f0d543b798f48176149582bb"
format_stamp: "Formatted at 2026-08-11 00:31:28 on dist-test-slave-fd7l"
server_key: "f3e73bd0aaf2ee15bf390866458c41e4"
server_key_iv: "dcc966ff59f58e24111dc7449b17b36e"
server_key_version: "encryptionkey@0"
I20260811 00:31:28.404633 23579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "5f6ffc20f0d543b798f48176149582bb"
format_stamp: "Formatted at 2026-08-11 00:31:28 on dist-test-slave-fd7l"
server_key: "f3e73bd0aaf2ee15bf390866458c41e4"
server_key_iv: "dcc966ff59f58e24111dc7449b17b36e"
server_key_version: "encryptionkey@0"
I20260811 00:31:28.411223 23579 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:31:28.416426 23597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:28.418206 23579 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:31:28.418416 23579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "5f6ffc20f0d543b798f48176149582bb"
format_stamp: "Formatted at 2026-08-11 00:31:28 on dist-test-slave-fd7l"
server_key: "f3e73bd0aaf2ee15bf390866458c41e4"
server_key_iv: "dcc966ff59f58e24111dc7449b17b36e"
server_key_version: "encryptionkey@0"
I20260811 00:31:28.418663 23579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:28.435871 23579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:28.441641 23579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:28.441998 23579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:28.443174 23579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:28.444825 23579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:28.444906 23579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:28.445065 23579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:28.445130 23579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:28.489924 23579 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:42193
I20260811 00:31:28.489975 23719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:42193 every 8 connection(s)
I20260811 00:31:28.491389 23579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:31:28.497498 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 23579
I20260811 00:31:28.497718 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:31:28.498320 2528 external_mini_cluster.cc:1428] Setting key d9cd11fa80d8c43f9513224c6fa66bce
Aug 11 00:31:28 dist-test-slave-fd7l krb5kdc[23299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408288, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:31:28.523241 23723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:28.495424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:34575 (local address 127.2.120.62:44433)
0811 00:31:28.496216 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:28.496222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:28.497623 (+ 1401us) server_negotiation.cc:407] Connection header received
0811 00:31:28.500162 (+ 2539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:28.500167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:28.500244 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:28.500379 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:28.502956 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:28.503840 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:28.506997 (+ 3157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:28.507493 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:28.507616 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:28.513688 (+ 6072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:28.513697 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:28.513701 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:28.513758 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:28.517665 (+ 3907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:28.518802 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:28.518814 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:28.518817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:28.519061 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:28.520130 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:28.520136 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:28.520138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:28.520680 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:28.520853 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:28.522474 (+ 1621us) server_negotiation.cc:299] Negotiation successful
0811 00:31:28.522826 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":442,"threads_started":1}
I20260811 00:31:28.525491 23725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:28.496294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44433 (local address 127.2.120.2:34575)
0811 00:31:28.497371 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:28.497446 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:28.499712 (+ 2266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:28.500732 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:28.500742 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:28.501574 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:28.502638 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:28.502673 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:28.504057 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:28.504065 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:28.506759 (+ 2694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:28.506785 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:28.507788 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:28.507798 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:28.508193 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:28.509157 (+ 964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:31:28.509196 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:28.513165 (+ 3969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:28.517950 (+ 4785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:28.517963 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:28.517983 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:28.518611 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:28.519336 (+ 725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:28.519347 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:28.519354 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:28.519856 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:28.520869 (+ 1013us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:28.520883 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:28.521722 (+ 839us) client_negotiation.cc:769] Sending connection context
0811 00:31:28.522230 (+ 508us) client_negotiation.cc:240] Negotiation successful
0811 00:31:28.523092 (+ 862us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":418,"threads_started":1}
I20260811 00:31:28.528005 23720 heartbeater.cc:347] Connected to a master server at 127.2.120.62:44433
I20260811 00:31:28.528457 23720 heartbeater.cc:464] Registering TS with master...
I20260811 00:31:28.529449 23720 heartbeater.cc:511] Master 127.2.120.62:44433 requested a full tablet report, sending...
I20260811 00:31:28.531971 23351 ts_manager.cc:194] Registered new tserver with Master: 5f6ffc20f0d543b798f48176149582bb (127.2.120.2:42193)
I20260811 00:31:28.533633 23351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:34575
WARNING: no policy specified for kudu/127.2.120.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
I20260811 00:31:28.585160 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:44433
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:38203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:31:28.867479 23573 heartbeater.cc:503] Master 127.2.120.62:44433 was elected leader, sending a full tablet report...
W20260811 00:31:29.007965 23728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:29.008376 23728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:29.008582 23728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:29.015939 23728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:29.016119 23728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:29.016319 23728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:31:29.027029 23728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:38203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:44433
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:29.028621 23728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:29.030134 23728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:29.045833 23734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:29.046425 23733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:29.046458 23736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:29.047626 23728 server_base.cc:1034] running on GCE node
I20260811 00:31:29.048472 23728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:29.049702 23728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:29.050971 23728 hybrid_clock.cc:648] HybridClock initialized: now 1786408289050920 us; error 33 us; skew 500 ppm
I20260811 00:31:29.051414 23728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:29 dist-test-slave-fd7l krb5kdc[23299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408289, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:29.055254 23728 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:31:29.057485 23728 webserver.cc:460] Webserver started at http://127.2.120.3:33273/ using document root <none> and password file <none>
I20260811 00:31:29.058362 23728 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:29.058495 23728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:29.058800 23728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:29.062199 23728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "aa1c3bfb290b4ef2a76a366e5a3b3d23"
format_stamp: "Formatted at 2026-08-11 00:31:29 on dist-test-slave-fd7l"
server_key: "c61823007d343647e9d4fc370aae548f"
server_key_iv: "84090144fdee1606cfaa2ac9cb2b2282"
server_key_version: "encryptionkey@0"
I20260811 00:31:29.063210 23728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "aa1c3bfb290b4ef2a76a366e5a3b3d23"
format_stamp: "Formatted at 2026-08-11 00:31:29 on dist-test-slave-fd7l"
server_key: "c61823007d343647e9d4fc370aae548f"
server_key_iv: "84090144fdee1606cfaa2ac9cb2b2282"
server_key_version: "encryptionkey@0"
I20260811 00:31:29.069509 23728 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260811 00:31:29.074568 23744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:29.076493 23728 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.002s
I20260811 00:31:29.076843 23728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "aa1c3bfb290b4ef2a76a366e5a3b3d23"
format_stamp: "Formatted at 2026-08-11 00:31:29 on dist-test-slave-fd7l"
server_key: "c61823007d343647e9d4fc370aae548f"
server_key_iv: "84090144fdee1606cfaa2ac9cb2b2282"
server_key_version: "encryptionkey@0"
I20260811 00:31:29.077108 23728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:29.115092 23728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:29.120371 23728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:29.120751 23728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:29.122037 23728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:29.123781 23728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:29.123886 23728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:29.124053 23728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:29.124128 23728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:29.163143 23728 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:37609
I20260811 00:31:29.163400 23872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:37609 every 8 connection(s)
I20260811 00:31:29.164651 23728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:31:29.166213 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 23728
I20260811 00:31:29.166435 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:31:29.166923 2528 external_mini_cluster.cc:1428] Setting key ec32092a571e1c6dc3fed61d20847ea5
Aug 11 00:31:29 dist-test-slave-fd7l krb5kdc[23299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408289, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:31:29.190325 23877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:29.167899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:56397 (local address 127.2.120.62:44433)
0811 00:31:29.168497 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:29.168532 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:29.169640 (+ 1108us) server_negotiation.cc:407] Connection header received
0811 00:31:29.171719 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:29.171724 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:29.171814 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:29.171949 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:29.173662 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.174472 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.176161 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.176687 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.176833 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:29.181339 (+ 4506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:29.181351 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:29.181358 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:29.181446 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:29.186005 (+ 4559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:29.186954 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:29.186959 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:29.186961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:29.187111 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:29.187799 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:29.187804 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:29.187807 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:29.188148 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:29.188293 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:29.189766 (+ 1473us) server_negotiation.cc:299] Negotiation successful
0811 00:31:29.189999 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":307,"threads_started":1}
I20260811 00:31:29.191923 23878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:29.168701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:44433 (local address 127.2.120.3:56397)
0811 00:31:29.169478 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:29.169524 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:29.171440 (+ 1916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:29.172205 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:29.172214 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:29.172844 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:29.173476 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:29.173493 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.174644 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.174656 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:29.175990 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:29.176004 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.176964 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.176975 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:29.177264 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:29.178065 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:31:29.178097 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:29.180726 (+ 2629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:29.186204 (+ 5478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:29.186214 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:29.186230 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:29.186767 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:29.187364 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:29.187368 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:29.187371 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:29.187611 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:29.188392 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:29.188403 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:29.189133 (+ 730us) client_negotiation.cc:769] Sending connection context
0811 00:31:29.189524 (+ 391us) client_negotiation.cc:240] Negotiation successful
0811 00:31:29.190163 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":329,"threads_started":1}
I20260811 00:31:29.194470 23875 heartbeater.cc:347] Connected to a master server at 127.2.120.62:44433
I20260811 00:31:29.194861 23875 heartbeater.cc:464] Registering TS with master...
I20260811 00:31:29.195971 23875 heartbeater.cc:511] Master 127.2.120.62:44433 requested a full tablet report, sending...
I20260811 00:31:29.198144 23351 ts_manager.cc:194] Registered new tserver with Master: aa1c3bfb290b4ef2a76a366e5a3b3d23 (127.2.120.3:37609)
I20260811 00:31:29.199126 23351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:56397
I20260811 00:31:29.210893 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:31:29.234995 23877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:29.215716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47438 (local address 127.2.120.62:44433)
0811 00:31:29.215966 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:29.215971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:29.216427 (+ 456us) server_negotiation.cc:407] Connection header received
0811 00:31:29.216645 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:29.216649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:29.216854 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:29.216965 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:29.219635 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.220559 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.222138 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.222971 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.223380 (+ 409us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:29.227168 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:29.227184 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:29.227188 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:29.227234 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:29.230423 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:29.231463 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:29.231468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:29.231471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:29.231631 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:29.232709 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:29.232713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:29.232715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:29.233105 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:29.233337 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:29.234130 (+ 793us) server_negotiation.cc:299] Negotiation successful
0811 00:31:29.234520 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260811 00:31:29.240411 23351 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:31:29.249341 23351 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:31:29.288133 23899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:29.271173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33130 (local address 127.2.120.2:42193)
0811 00:31:29.277101 (+ 5928us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:29.277114 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:29.278863 (+ 1749us) server_negotiation.cc:407] Connection header received
0811 00:31:29.278987 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:29.278996 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:29.279214 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:29.279424 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:29.280471 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.281891 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.285124 (+ 3233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.286528 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.286684 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:29.286919 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:29.287530 (+ 611us) server_negotiation.cc:299] Negotiation successful
0811 00:31:29.287731 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5720,"thread_start_us":5566,"threads_started":1}
I20260811 00:31:29.289036 23894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:29.269501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42950 (local address 127.2.120.3:37609)
0811 00:31:29.276789 (+ 7288us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:29.276803 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:29.276849 (+ 46us) server_negotiation.cc:407] Connection header received
0811 00:31:29.276996 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:29.277009 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:29.277385 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:29.277685 (+ 300us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:29.279199 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.281200 (+ 2001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.285986 (+ 4786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.287373 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.287585 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:29.287804 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:29.288410 (+ 606us) server_negotiation.cc:299] Negotiation successful
0811 00:31:29.288624 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7038,"thread_start_us":6750,"threads_started":1}
I20260811 00:31:29.289702 23898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:29.269998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:36125 (local address 127.0.0.1:46166)
0811 00:31:29.277666 (+ 7668us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:29.277689 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:29.277839 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:29.278912 (+ 1073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:29.278916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:29.278944 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:29.279710 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:29.279723 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.281805 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.281817 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:29.285768 (+ 3951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:29.285891 (+ 123us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.288693 (+ 2802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.288700 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:29.288900 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:29.288945 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 00:31:29.289055 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 00:31:29.289235 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7515,"mutex_wait_us":99,"thread_start_us":8197,"threads_started":1}
I20260811 00:31:29.289913 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:29.269076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:37609 (local address 127.0.0.1:42950)
0811 00:31:29.276209 (+ 7133us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:29.276292 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:29.276614 (+ 322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:29.278234 (+ 1620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:29.278243 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:29.278282 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:29.278924 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:29.278957 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.281379 (+ 2422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.281385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:29.285405 (+ 4020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:29.285429 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.287960 (+ 2531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.287967 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:29.288152 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:29.288201 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 00:31:29.289292 (+ 1091us) client_negotiation.cc:240] Negotiation successful
0811 00:31:29.289466 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":101,"thread_start_us":386,"threads_started":1}
I20260811 00:31:29.290167 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:29.270600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:42193 (local address 127.0.0.1:33130)
0811 00:31:29.278542 (+ 7942us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:29.278566 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:29.278704 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:29.279735 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:29.279739 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:29.279765 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:29.280276 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:29.280290 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.282060 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.282071 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:29.284928 (+ 2857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:29.284944 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.286804 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.286812 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:29.287079 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:29.287153 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 00:31:29.287447 (+ 294us) client_negotiation.cc:240] Negotiation successful
0811 00:31:29.287879 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7771,"mutex_wait_us":4,"thread_start_us":6746,"threads_started":1}
I20260811 00:31:29.290865 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:29.270389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46166 (local address 127.2.120.1:36125)
0811 00:31:29.278174 (+ 7785us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:29.278187 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:29.278215 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:31:29.278322 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:29.278334 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:29.278586 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:29.279096 (+ 510us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:29.279949 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.281639 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.286163 (+ 4524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.288238 (+ 2075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.288413 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:29.290149 (+ 1736us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:29.290248 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 00:31:29.290505 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7541,"thread_start_us":7543,"threads_started":1}
I20260811 00:31:29.294250 23800 tablet_service.cc:1467] Processing CreateTablet for tablet e9e11f20e8294e0685177ce58465e28b (DEFAULT_TABLE table=test-table [id=d99e3aa70cee42b4afb14685fa3c02af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:31:29.294260 23496 tablet_service.cc:1467] Processing CreateTablet for tablet e9e11f20e8294e0685177ce58465e28b (DEFAULT_TABLE table=test-table [id=d99e3aa70cee42b4afb14685fa3c02af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:31:29.294261 23649 tablet_service.cc:1467] Processing CreateTablet for tablet e9e11f20e8294e0685177ce58465e28b (DEFAULT_TABLE table=test-table [id=d99e3aa70cee42b4afb14685fa3c02af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:31:29.295800 23496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9e11f20e8294e0685177ce58465e28b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:29.295809 23649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9e11f20e8294e0685177ce58465e28b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:29.296068 23800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9e11f20e8294e0685177ce58465e28b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:29.311933 23900 tablet_bootstrap.cc:492] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23: Bootstrap starting.
I20260811 00:31:29.315615 23900 tablet_bootstrap.cc:654] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:29.316526 23903 tablet_bootstrap.cc:492] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6: Bootstrap starting.
I20260811 00:31:29.316648 23902 tablet_bootstrap.cc:492] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb: Bootstrap starting.
I20260811 00:31:29.317377 23900 log.cc:824] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:29.321645 23903 tablet_bootstrap.cc:654] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:29.321667 23902 tablet_bootstrap.cc:654] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:29.322775 23900 tablet_bootstrap.cc:492] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23: No bootstrap required, opened a new log
I20260811 00:31:29.323164 23900 ts_tablet_manager.cc:1397] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.002s
I20260811 00:31:29.323199 23903 log.cc:824] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:29.323645 23902 log.cc:824] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:29.327389 23902 tablet_bootstrap.cc:492] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb: No bootstrap required, opened a new log
I20260811 00:31:29.327389 23903 tablet_bootstrap.cc:492] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6: No bootstrap required, opened a new log
I20260811 00:31:29.327752 23903 ts_tablet_manager.cc:1397] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.003s
I20260811 00:31:29.327752 23902 ts_tablet_manager.cc:1397] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 00:31:29.332449 23900 raft_consensus.cc:348] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa1c3bfb290b4ef2a76a366e5a3b3d23" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37609 } } peers { permanent_uuid: "5f6ffc20f0d543b798f48176149582bb" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42193 } } peers { permanent_uuid: "14e3ceb6d95248a9a5ea8598a3faa5b6" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 36125 } }
I20260811 00:31:29.333230 23900 raft_consensus.cc:374] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:29.333376 23900 raft_consensus.cc:729] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa1c3bfb290b4ef2a76a366e5a3b3d23, State: Initialized, Role: FOLLOWER
I20260811 00:31:29.334179 23900 consensus_queue.cc:260] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa1c3bfb290b4ef2a76a366e5a3b3d23" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37609 } } peers { permanent_uuid: "5f6ffc20f0d543b798f48176149582bb" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42193 } } peers { permanent_uuid: "14e3ceb6d95248a9a5ea8598a3faa5b6" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 36125 } }
I20260811 00:31:29.335736 23875 heartbeater.cc:503] Master 127.2.120.62:44433 was elected leader, sending a full tablet report...
I20260811 00:31:29.336687 23900 ts_tablet_manager.cc:1428] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23: Time spent starting tablet: real 0.013s user 0.008s sys 0.007s
I20260811 00:31:29.339834 23903 raft_consensus.cc:348] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa1c3bfb290b4ef2a76a366e5a3b3d23" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37609 } } peers { permanent_uuid: "5f6ffc20f0d543b798f48176149582bb" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42193 } } peers { permanent_uuid: "14e3ceb6d95248a9a5ea8598a3faa5b6" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 36125 } }
I20260811 00:31:29.339834 23902 raft_consensus.cc:348] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa1c3bfb290b4ef2a76a366e5a3b3d23" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37609 } } peers { permanent_uuid: "5f6ffc20f0d543b798f48176149582bb" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42193 } } peers { permanent_uuid: "14e3ceb6d95248a9a5ea8598a3faa5b6" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 36125 } }
I20260811 00:31:29.340384 23903 raft_consensus.cc:374] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:29.340384 23902 raft_consensus.cc:374] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:29.340566 23903 raft_consensus.cc:729] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14e3ceb6d95248a9a5ea8598a3faa5b6, State: Initialized, Role: FOLLOWER
I20260811 00:31:29.340577 23902 raft_consensus.cc:729] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f6ffc20f0d543b798f48176149582bb, State: Initialized, Role: FOLLOWER
I20260811 00:31:29.341341 23903 consensus_queue.cc:260] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa1c3bfb290b4ef2a76a366e5a3b3d23" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37609 } } peers { permanent_uuid: "5f6ffc20f0d543b798f48176149582bb" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42193 } } peers { permanent_uuid: "14e3ceb6d95248a9a5ea8598a3faa5b6" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 36125 } }
I20260811 00:31:29.341346 23902 consensus_queue.cc:260] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa1c3bfb290b4ef2a76a366e5a3b3d23" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37609 } } peers { permanent_uuid: "5f6ffc20f0d543b798f48176149582bb" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42193 } } peers { permanent_uuid: "14e3ceb6d95248a9a5ea8598a3faa5b6" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 36125 } }
I20260811 00:31:29.343335 23720 heartbeater.cc:503] Master 127.2.120.62:44433 was elected leader, sending a full tablet report...
I20260811 00:31:29.343765 23903 ts_tablet_manager.cc:1428] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6: Time spent starting tablet: real 0.016s user 0.016s sys 0.001s
I20260811 00:31:29.343775 23902 ts_tablet_manager.cc:1428] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
I20260811 00:31:29.394865 23907 raft_consensus.cc:482] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:31:29.395263 23907 raft_consensus.cc:504] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa1c3bfb290b4ef2a76a366e5a3b3d23" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37609 } } peers { permanent_uuid: "5f6ffc20f0d543b798f48176149582bb" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42193 } } peers { permanent_uuid: "14e3ceb6d95248a9a5ea8598a3faa5b6" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 36125 } }
I20260811 00:31:29.397389 23907 leader_election.cc:288] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f6ffc20f0d543b798f48176149582bb (127.2.120.2:42193), 14e3ceb6d95248a9a5ea8598a3faa5b6 (127.2.120.1:36125)
I20260811 00:31:29.404429 23878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:29.397866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:42193 (local address 127.2.120.3:51609)
0811 00:31:29.398124 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:29.398150 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:29.398324 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:29.398837 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:29.398841 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:29.398873 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:29.399143 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:29.399154 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.400299 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.400307 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:29.402220 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:29.402238 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.403590 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.403596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:29.403802 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:29.403829 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 00:31:29.403961 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 00:31:29.404122 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 00:31:29.404428 23899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:29.397867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:51609 (local address 127.2.120.2:42193)
0811 00:31:29.398112 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:29.398119 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:29.398257 (+ 138us) server_negotiation.cc:407] Connection header received
0811 00:31:29.398542 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:29.398547 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:29.398643 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:29.398768 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:29.399304 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.400142 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.402412 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.403371 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.403573 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:29.403678 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:29.403984 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 00:31:29.404124 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:31:29.406481 23910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:29.397865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:36125 (local address 127.2.120.3:49625)
0811 00:31:29.398860 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:29.398881 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:29.399017 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:29.399580 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:29.399587 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:29.399639 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:29.400076 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:29.400088 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.401774 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.401779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:29.403391 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:29.403406 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.405749 (+ 2343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.405754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:29.405896 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:29.405925 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 00:31:29.406028 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 00:31:29.406136 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":857,"mutex_wait_us":245,"thread_start_us":384,"threads_started":1}
I20260811 00:31:29.406191 23671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9e11f20e8294e0685177ce58465e28b" candidate_uuid: "aa1c3bfb290b4ef2a76a366e5a3b3d23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f6ffc20f0d543b798f48176149582bb" is_pre_election: true
I20260811 00:31:29.406664 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:29.398064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:49625 (local address 127.2.120.1:36125)
0811 00:31:29.398331 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:29.398339 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:29.398989 (+ 650us) server_negotiation.cc:407] Connection header received
0811 00:31:29.399169 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:29.399176 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:29.399287 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:29.399393 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:29.400404 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.401548 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.403619 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.405444 (+ 1825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.405688 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:29.405823 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:29.406147 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 00:31:29.406365 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 00:31:29.407017 23671 raft_consensus.cc:2454] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa1c3bfb290b4ef2a76a366e5a3b3d23 in term 0.
I20260811 00:31:29.408041 23746 leader_election.cc:302] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f6ffc20f0d543b798f48176149582bb, aa1c3bfb290b4ef2a76a366e5a3b3d23; no voters:
I20260811 00:31:29.408102 23519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9e11f20e8294e0685177ce58465e28b" candidate_uuid: "aa1c3bfb290b4ef2a76a366e5a3b3d23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14e3ceb6d95248a9a5ea8598a3faa5b6" is_pre_election: true
I20260811 00:31:29.408805 23519 raft_consensus.cc:2454] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa1c3bfb290b4ef2a76a366e5a3b3d23 in term 0.
I20260811 00:31:29.408842 23907 raft_consensus.cc:2781] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:31:29.409145 23907 raft_consensus.cc:482] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:31:29.409566 23907 raft_consensus.cc:3037] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:29.412850 23907 raft_consensus.cc:504] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa1c3bfb290b4ef2a76a366e5a3b3d23" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37609 } } peers { permanent_uuid: "5f6ffc20f0d543b798f48176149582bb" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42193 } } peers { permanent_uuid: "14e3ceb6d95248a9a5ea8598a3faa5b6" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 36125 } }
I20260811 00:31:29.413623 23907 leader_election.cc:288] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [CANDIDATE]: Term 1 election: Requested vote from peers 5f6ffc20f0d543b798f48176149582bb (127.2.120.2:42193), 14e3ceb6d95248a9a5ea8598a3faa5b6 (127.2.120.1:36125)
I20260811 00:31:29.414525 23671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9e11f20e8294e0685177ce58465e28b" candidate_uuid: "aa1c3bfb290b4ef2a76a366e5a3b3d23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f6ffc20f0d543b798f48176149582bb"
I20260811 00:31:29.414505 23519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9e11f20e8294e0685177ce58465e28b" candidate_uuid: "aa1c3bfb290b4ef2a76a366e5a3b3d23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14e3ceb6d95248a9a5ea8598a3faa5b6"
I20260811 00:31:29.414799 23671 raft_consensus.cc:3037] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:29.414798 23519 raft_consensus.cc:3037] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6 [term 0 FOLLOWER]: Advancing to term 1
W20260811 00:31:29.416729 23876 tablet.cc:2369] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:31:29.418601 23671 raft_consensus.cc:2454] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa1c3bfb290b4ef2a76a366e5a3b3d23 in term 1.
I20260811 00:31:29.418601 23519 raft_consensus.cc:2454] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa1c3bfb290b4ef2a76a366e5a3b3d23 in term 1.
I20260811 00:31:29.419175 23746 leader_election.cc:302] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f6ffc20f0d543b798f48176149582bb, aa1c3bfb290b4ef2a76a366e5a3b3d23; no voters:
I20260811 00:31:29.419576 23907 raft_consensus.cc:2781] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:29.420614 23907 raft_consensus.cc:686] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [term 1 LEADER]: Becoming Leader. State: Replica: aa1c3bfb290b4ef2a76a366e5a3b3d23, State: Running, Role: LEADER
I20260811 00:31:29.421314 23907 consensus_queue.cc:237] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa1c3bfb290b4ef2a76a366e5a3b3d23" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37609 } } peers { permanent_uuid: "5f6ffc20f0d543b798f48176149582bb" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42193 } } peers { permanent_uuid: "14e3ceb6d95248a9a5ea8598a3faa5b6" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 36125 } }
I20260811 00:31:29.427749 23348 catalog_manager.cc:5495] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa1c3bfb290b4ef2a76a366e5a3b3d23 (127.2.120.3). New cstate: current_term: 1 leader_uuid: "aa1c3bfb290b4ef2a76a366e5a3b3d23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa1c3bfb290b4ef2a76a366e5a3b3d23" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f6ffc20f0d543b798f48176149582bb" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14e3ceb6d95248a9a5ea8598a3faa5b6" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 36125 } health_report { overall_health: UNKNOWN } } }
I20260811 00:31:29.460836 23894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:29.451505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42962 (local address 127.2.120.3:37609)
0811 00:31:29.451760 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:29.451767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:29.451843 (+ 76us) server_negotiation.cc:407] Connection header received
0811 00:31:29.452126 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:29.452130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:29.452229 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:29.452415 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:31:29.453523 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.454592 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.456290 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.456923 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.457057 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:29.458155 (+ 1098us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:31:29.459081 (+ 926us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:31:29.459388 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:29.459972 (+ 584us) server_negotiation.cc:299] Negotiation successful
0811 00:31:29.460095 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 00:31:29.475189 23519 raft_consensus.cc:1261] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6 [term 1 FOLLOWER]: Refusing update from remote peer aa1c3bfb290b4ef2a76a366e5a3b3d23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:31:29.475184 23671 raft_consensus.cc:1261] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb [term 1 FOLLOWER]: Refusing update from remote peer aa1c3bfb290b4ef2a76a366e5a3b3d23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:31:29.476735 23911 consensus_queue.cc:1035] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14e3ceb6d95248a9a5ea8598a3faa5b6" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 36125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:31:29.477167 23907 consensus_queue.cc:1035] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f6ffc20f0d543b798f48176149582bb" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 42193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 00:31:29.494181 23721 tablet.cc:2369] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:31:29.502084 23915 mvcc.cc:205] Tried to move back new op lower bound from 7317128353675214848 to 7317128353474834432. Current Snapshot: MvccSnapshot[applied={T|T < 7317128353675214848}]
I20260811 00:31:29.530944 23348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:31:29.531317 23348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:31:29.535897 23348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27c1e565ec9d48148d38672c4389e78f: Sending DeleteTablet for 3 replicas of tablet e9e11f20e8294e0685177ce58465e28b
I20260811 00:31:29.537519 23800 tablet_service.cc:1514] Processing DeleteTablet for tablet e9e11f20e8294e0685177ce58465e28b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:31:29 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:42950
I20260811 00:31:29.537635 23496 tablet_service.cc:1514] Processing DeleteTablet for tablet e9e11f20e8294e0685177ce58465e28b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:31:29 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:46166
I20260811 00:31:29.538683 23927 tablet_replica.cc:330] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23: stopping tablet replica
I20260811 00:31:29.538864 23649 tablet_service.cc:1514] Processing DeleteTablet for tablet e9e11f20e8294e0685177ce58465e28b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:31:29 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:33130
I20260811 00:31:29.539113 23928 tablet_replica.cc:330] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6: stopping tablet replica
I20260811 00:31:29.539436 23927 raft_consensus.cc:2229] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:31:29.539933 23928 raft_consensus.cc:2229] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:31:29.540326 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 23422
I20260811 00:31:29.540611 23927 raft_consensus.cc:2258] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:31:29.540637 23928 raft_consensus.cc:2258] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:31:29.543812 23927 ts_tablet_manager.cc:1905] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:31:29.544718 23928 ts_tablet_manager.cc:1905] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:31:29.552446 23928 ts_tablet_manager.cc:1918] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:31:29.552721 23928 log.cc:1196] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-0/wal/wals/e9e11f20e8294e0685177ce58465e28b
I20260811 00:31:29.553363 23928 ts_tablet_manager.cc:1939] T e9e11f20e8294e0685177ce58465e28b P 14e3ceb6d95248a9a5ea8598a3faa5b6: Deleting consensus metadata
I20260811 00:31:29.554414 23927 ts_tablet_manager.cc:1918] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:31:29.554770 23927 log.cc:1196] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/e9e11f20e8294e0685177ce58465e28b
I20260811 00:31:29.555635 23927 ts_tablet_manager.cc:1939] T e9e11f20e8294e0685177ce58465e28b P aa1c3bfb290b4ef2a76a366e5a3b3d23: Deleting consensus metadata
I20260811 00:31:29.555826 23332 catalog_manager.cc:4841] TS 14e3ceb6d95248a9a5ea8598a3faa5b6 (127.2.120.1:36125): tablet e9e11f20e8294e0685177ce58465e28b (table test-table [id=d99e3aa70cee42b4afb14685fa3c02af]) successfully deleted
I20260811 00:31:29.556945 23929 tablet_replica.cc:330] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb: stopping tablet replica
I20260811 00:31:29.557657 23929 raft_consensus.cc:2229] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:31:29.558528 23330 catalog_manager.cc:4841] TS aa1c3bfb290b4ef2a76a366e5a3b3d23 (127.2.120.3:37609): tablet e9e11f20e8294e0685177ce58465e28b (table test-table [id=d99e3aa70cee42b4afb14685fa3c02af]) successfully deleted
I20260811 00:31:29.558974 23929 raft_consensus.cc:2258] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:31:29.561137 23929 ts_tablet_manager.cc:1905] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 00:31:29 dist-test-slave-fd7l krb5kdc[23299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:31:29.569538 23929 ts_tablet_manager.cc:1918] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:31:29.569789 23929 log.cc:1196] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/e9e11f20e8294e0685177ce58465e28b
I20260811 00:31:29.570514 23929 ts_tablet_manager.cc:1939] T e9e11f20e8294e0685177ce58465e28b P 5f6ffc20f0d543b798f48176149582bb: Deleting consensus metadata
I20260811 00:31:29.572902 23330 catalog_manager.cc:4841] TS 5f6ffc20f0d543b798f48176149582bb (127.2.120.2:42193): tablet e9e11f20e8294e0685177ce58465e28b (table test-table [id=d99e3aa70cee42b4afb14685fa3c02af]) successfully deleted
I20260811 00:31:29.578569 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:29.542536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46176 (local address 127.2.120.1:36125)
0811 00:31:29.542743 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:29.542751 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:29.556177 (+ 13426us) server_negotiation.cc:407] Connection header received
0811 00:31:29.556386 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:29.556392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:29.556482 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:29.556638 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:29.558985 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.560087 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.562335 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.563018 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.563533 (+ 515us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:29.570093 (+ 6560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:29.570123 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:29.570150 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:29.570216 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:29.574277 (+ 4061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:29.575362 (+ 1085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:29.575371 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:29.575373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:29.575532 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:29.576153 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:29.576162 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:29.576165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:29.577086 (+ 921us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:29.577365 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:29.577932 (+ 567us) server_negotiation.cc:299] Negotiation successful
0811 00:31:29.578136 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 00:31:29.579325 23566 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:29.692817 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 23422
I20260811 00:31:29.719399 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 23579
Aug 11 00:31:29 dist-test-slave-fd7l krb5kdc[23299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.2@KRBTEST.COM
I20260811 00:31:29.740231 23899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:29.720405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33140 (local address 127.2.120.2:42193)
0811 00:31:29.720618 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:29.720622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:29.720643 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:31:29.720795 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:29.720798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:29.720867 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:29.720955 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:29.723521 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.724726 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.727385 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.727855 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.727986 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:29.733728 (+ 5742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:29.733749 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:29.733768 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:29.733815 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:29.737017 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:29.737890 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:29.737898 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:29.737901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:29.738047 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:29.738452 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:29.738458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:29.738461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:29.738926 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:29.739264 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:29.739612 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 00:31:29.739857 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:31:29.741128 23715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:29.842909 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 23579
I20260811 00:31:29.874513 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 23728
Aug 11 00:31:29 dist-test-slave-fd7l krb5kdc[23299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.3@KRBTEST.COM
I20260811 00:31:29.897131 23894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:29.875869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42974 (local address 127.2.120.3:37609)
0811 00:31:29.876128 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:29.876140 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:29.876171 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:31:29.876332 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:29.876338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:29.876448 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:29.876664 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:29.878370 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.879694 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.881908 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:29.882771 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:29.882944 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:29.888297 (+ 5353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:29.888316 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:29.888341 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:29.888396 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:29.893287 (+ 4891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:29.894354 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:29.894365 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:29.894369 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:29.894597 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:29.895087 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:29.895092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:29.895096 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:29.895646 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:29.896024 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:29.896403 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 00:31:29.896706 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260811 00:31:29.898036 23868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:30.045766 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 23728
I20260811 00:31:30.075007 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 23312
I20260811 00:31:30.077124 23385 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:30.200687 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 23312
2026-08-11T00:31:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5459 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T00:31:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:31:30Z Disabled control of system clock
I20260811 00:31:30.322826 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:39989
--webserver_interface=127.2.120.62
--webserver_port=0
--builtin_ntp_servers=127.2.120.20:35351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:39989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 00:31:30.772408 23940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:30.772886 23940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:30.773229 23940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:30.783206 23940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:31:30.783447 23940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:30.783499 23940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:30.783635 23940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:31:30.783718 23940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:31:30.796654 23940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:35351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:39989
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:39989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:30.798326 23940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:30.801517 23940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:30.817488 23950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:30.817696 23945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:30.817696 23946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:30.819046 23940 server_base.cc:1034] running on GCE node
I20260811 00:31:30.820235 23940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:30.822338 23940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:30.823948 23940 hybrid_clock.cc:648] HybridClock initialized: now 1786408290823876 us; error 68 us; skew 500 ppm
I20260811 00:31:30.824561 23940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:31:30.827947 23940 webserver.cc:460] Webserver started at http://127.2.120.62:42753/ using document root <none> and password file <none>
I20260811 00:31:30.829003 23940 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:30.829170 23940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:30.829550 23940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:30.833254 23940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "7e690188a5714aa5a51419447cd85e65"
format_stamp: "Formatted at 2026-08-11 00:31:30 on dist-test-slave-fd7l"
server_key: "31dcc74a002d4315e1bf3bf7495a9198"
server_key_iv: "ccdf73df679e59773b2e71f6710e6fc9"
server_key_version: "encryptionkey@0"
I20260811 00:31:30.834335 23940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "7e690188a5714aa5a51419447cd85e65"
format_stamp: "Formatted at 2026-08-11 00:31:30 on dist-test-slave-fd7l"
server_key: "31dcc74a002d4315e1bf3bf7495a9198"
server_key_iv: "ccdf73df679e59773b2e71f6710e6fc9"
server_key_version: "encryptionkey@0"
I20260811 00:31:30.841697 23940 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 00:31:30.847189 23956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:30.849222 23940 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.004s
I20260811 00:31:30.849539 23940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "7e690188a5714aa5a51419447cd85e65"
format_stamp: "Formatted at 2026-08-11 00:31:30 on dist-test-slave-fd7l"
server_key: "31dcc74a002d4315e1bf3bf7495a9198"
server_key_iv: "ccdf73df679e59773b2e71f6710e6fc9"
server_key_version: "encryptionkey@0"
I20260811 00:31:30.849851 23940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:30.872825 23940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:30.874361 23940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:30.874872 23940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:30.901005 24016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:39989 every 8 connection(s)
I20260811 00:31:30.900962 23940 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:39989
I20260811 00:31:30.903098 23940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:31:30.908550 24017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:30.909281 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 23940
I20260811 00:31:30.909499 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:31:30.910204 2528 external_mini_cluster.cc:1428] Setting key 1bf6ed602a07693fcb9511dd6370bbb2
I20260811 00:31:30.928399 24017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65: Bootstrap starting.
I20260811 00:31:30.930889 24020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:30.913457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57174 (local address 127.2.120.62:39989)
0811 00:31:30.915238 (+ 1781us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:30.915259 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:30.915328 (+ 69us) server_negotiation.cc:407] Connection header received
0811 00:31:30.917633 (+ 2305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:30.917675 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:30.918715 (+ 1040us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:30.919193 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:30.920256 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:30.921935 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:30.924473 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:30.925544 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:30.925760 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:30.927565 (+ 1805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:30.927578 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:30.927603 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:30.928383 (+ 780us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:30.928498 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:30.928613 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:30.928803 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:30.929094 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 00:31:30.929578 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1074,"thread_start_us":708,"threads_started":1}
I20260811 00:31:30.933246 24017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:30.934729 24017 log.cc:824] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:30.938987 24017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65: No bootstrap required, opened a new log
I20260811 00:31:30.947947 24017 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e690188a5714aa5a51419447cd85e65" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39989 } }
I20260811 00:31:30.948403 24017 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:30.948479 24017 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e690188a5714aa5a51419447cd85e65, State: Initialized, Role: FOLLOWER
I20260811 00:31:30.949234 24017 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e690188a5714aa5a51419447cd85e65" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39989 } }
I20260811 00:31:30.949471 24017 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:31:30.949576 24017 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:31:30.949728 24017 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:30.952567 24017 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e690188a5714aa5a51419447cd85e65" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39989 } }
I20260811 00:31:30.953181 24017 leader_election.cc:302] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e690188a5714aa5a51419447cd85e65; no voters:
I20260811 00:31:30.953859 24017 leader_election.cc:288] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:31:30.954066 24023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:30.954948 24023 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65 [term 1 LEADER]: Becoming Leader. State: Replica: 7e690188a5714aa5a51419447cd85e65, State: Running, Role: LEADER
I20260811 00:31:30.955600 24023 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e690188a5714aa5a51419447cd85e65" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39989 } }
I20260811 00:31:30.956382 24017 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:31:30.958767 24024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e690188a5714aa5a51419447cd85e65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e690188a5714aa5a51419447cd85e65" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39989 } } }
I20260811 00:31:30.959856 24024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65 [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:30.959457 24025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e690188a5714aa5a51419447cd85e65. Latest consensus state: current_term: 1 leader_uuid: "7e690188a5714aa5a51419447cd85e65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e690188a5714aa5a51419447cd85e65" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39989 } } }
I20260811 00:31:30.960137 24025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65 [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:30.961557 24030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:31:30.968226 24030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:31:30.984439 24030 catalog_manager.cc:1347] Generated new cluster ID: 2e18985eecae4cae98d8ec80b789395d
I20260811 00:31:30.984819 24030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:31:30.999869 24030 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:31:31.001972 24030 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:31:31.015098 24030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65: Generated new TSK 0
I20260811 00:31:31.016642 24030 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:31:31.035254 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:39989
--builtin_ntp_servers=127.2.120.20:35351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 00:31:31.493021 24045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:31.493371 24045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:31.493530 24045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:31.501704 24045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:31.501847 24045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:31.502034 24045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:31:31.512467 24045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:35351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:39989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:31.514146 24045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:31.516422 24045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:31.532804 24052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:31.532704 24053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:31.532728 24055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:31.533761 24045 server_base.cc:1034] running on GCE node
I20260811 00:31:31.534523 24045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:31.535694 24045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:31.536968 24045 hybrid_clock.cc:648] HybridClock initialized: now 1786408291536894 us; error 66 us; skew 500 ppm
I20260811 00:31:31.537361 24045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:31:31.540230 24045 webserver.cc:460] Webserver started at http://127.2.120.1:44893/ using document root <none> and password file <none>
I20260811 00:31:31.541174 24045 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:31.541332 24045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:31.541714 24045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:31.545351 24045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "26cb7439e4bb4593a63a551577f1ba22"
format_stamp: "Formatted at 2026-08-11 00:31:31 on dist-test-slave-fd7l"
server_key: "d3750fc3b75ca7e302fc5649fb2c3df2"
server_key_iv: "a741f8a441f7540ae7b066322822a2b7"
server_key_version: "encryptionkey@0"
I20260811 00:31:31.546968 24045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "26cb7439e4bb4593a63a551577f1ba22"
format_stamp: "Formatted at 2026-08-11 00:31:31 on dist-test-slave-fd7l"
server_key: "d3750fc3b75ca7e302fc5649fb2c3df2"
server_key_iv: "a741f8a441f7540ae7b066322822a2b7"
server_key_version: "encryptionkey@0"
I20260811 00:31:31.556106 24045 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260811 00:31:31.562642 24065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:31.565303 24045 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.003s
I20260811 00:31:31.565745 24045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "26cb7439e4bb4593a63a551577f1ba22"
format_stamp: "Formatted at 2026-08-11 00:31:31 on dist-test-slave-fd7l"
server_key: "d3750fc3b75ca7e302fc5649fb2c3df2"
server_key_iv: "a741f8a441f7540ae7b066322822a2b7"
server_key_version: "encryptionkey@0"
I20260811 00:31:31.566241 24045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:31.598124 24045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:31.599207 24045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:31.599582 24045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:31.600921 24045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:31.602815 24045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:31.602936 24045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:31.603098 24045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:31.603219 24045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:31.655179 24045 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:37999
I20260811 00:31:31.655179 24197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:37999 every 8 connection(s)
I20260811 00:31:31.657060 24045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:31:31.663971 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 24045
I20260811 00:31:31.664258 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:31:31.665273 2528 external_mini_cluster.cc:1428] Setting key f95f25e99d768dc928d67c63d10617d8
I20260811 00:31:31.669482 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:39989
--builtin_ntp_servers=127.2.120.20:35351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 00:31:31.678256 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:31.660467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:49515 (local address 127.2.120.62:39989)
0811 00:31:31.661105 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:31.661113 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:31.663217 (+ 2104us) server_negotiation.cc:407] Connection header received
0811 00:31:31.666869 (+ 3652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:31.666876 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:31.667030 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:31.667186 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:31.669054 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:31.670094 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:31.673634 (+ 3540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:31.674150 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:31.674308 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:31.677081 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:31.677089 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:31.677093 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:31.677134 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:31.677189 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:31.677211 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:31.677322 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:31.677756 (+ 434us) server_negotiation.cc:299] Negotiation successful
0811 00:31:31.677924 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":265,"threads_started":1}
I20260811 00:31:31.680123 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:31.661113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:39989 (local address 127.2.120.1:49515)
0811 00:31:31.662916 (+ 1803us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:31.662981 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:31.666398 (+ 3417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:31.667518 (+ 1120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:31.667529 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:31.667666 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:31.668728 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:31.668761 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:31.670380 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:31.670390 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:31.673288 (+ 2898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:31.673322 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:31.674514 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:31.674525 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:31.675011 (+ 486us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:31.676381 (+ 1370us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:31:31.676388 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:31.676487 (+ 99us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:31:31.676688 (+ 201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:31.677365 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:31.677383 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:31.677395 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 00:31:31.677657 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 00:31:31.678287 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"thread_start_us":543,"threads_started":1}
I20260811 00:31:31.682606 24198 heartbeater.cc:347] Connected to a master server at 127.2.120.62:39989
I20260811 00:31:31.683073 24198 heartbeater.cc:464] Registering TS with master...
I20260811 00:31:31.684237 24198 heartbeater.cc:511] Master 127.2.120.62:39989 requested a full tablet report, sending...
I20260811 00:31:31.687404 23977 ts_manager.cc:194] Registered new tserver with Master: 26cb7439e4bb4593a63a551577f1ba22 (127.2.120.1:37999)
I20260811 00:31:31.689322 23977 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.120.1:49515
W20260811 00:31:32.103978 24202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:32.104336 24202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:32.104514 24202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:32.113616 24202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:32.113740 24202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:32.113919 24202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:31:32.123828 24202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:35351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:39989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:32.125391 24202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:32.127480 24202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:32.148799 24210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:32.148828 24211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:32.149044 24213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:32.149555 24202 server_base.cc:1034] running on GCE node
I20260811 00:31:32.150741 24202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:32.152390 24202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:32.153764 24202 hybrid_clock.cc:648] HybridClock initialized: now 1786408292153666 us; error 94 us; skew 500 ppm
I20260811 00:31:32.154371 24202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:31:32.163643 24202 webserver.cc:460] Webserver started at http://127.2.120.2:44355/ using document root <none> and password file <none>
I20260811 00:31:32.165078 24202 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:32.165278 24202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:32.165849 24202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:32.172703 24202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c"
format_stamp: "Formatted at 2026-08-11 00:31:32 on dist-test-slave-fd7l"
server_key: "6723c19f548a34167433b50847025fa8"
server_key_iv: "45c390addfdb82d3c7bcde7c60539c5f"
server_key_version: "encryptionkey@0"
I20260811 00:31:32.174491 24202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c"
format_stamp: "Formatted at 2026-08-11 00:31:32 on dist-test-slave-fd7l"
server_key: "6723c19f548a34167433b50847025fa8"
server_key_iv: "45c390addfdb82d3c7bcde7c60539c5f"
server_key_version: "encryptionkey@0"
I20260811 00:31:32.184619 24202 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.001s sys 0.010s
I20260811 00:31:32.192090 24219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:32.194731 24202 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.001s
I20260811 00:31:32.195055 24202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c"
format_stamp: "Formatted at 2026-08-11 00:31:32 on dist-test-slave-fd7l"
server_key: "6723c19f548a34167433b50847025fa8"
server_key_iv: "45c390addfdb82d3c7bcde7c60539c5f"
server_key_version: "encryptionkey@0"
I20260811 00:31:32.195458 24202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:32.223516 24202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:32.224448 24202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:32.224802 24202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:32.226390 24202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:32.228153 24202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:32.228273 24202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:32.228365 24202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:32.228422 24202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:32.270138 24202 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:37423
I20260811 00:31:32.270187 24347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:37423 every 8 connection(s)
I20260811 00:31:32.271757 24202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:31:32.275038 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 24202
I20260811 00:31:32.275200 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:31:32.275696 2528 external_mini_cluster.cc:1428] Setting key 4d09ebb57ea01e3c5e199f226d287582
I20260811 00:31:32.280604 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:39989
--builtin_ntp_servers=127.2.120.20:35351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 00:31:32.289207 24350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:32.276413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:43873 (local address 127.2.120.62:39989)
0811 00:31:32.277373 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:32.277380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:32.280639 (+ 3259us) server_negotiation.cc:407] Connection header received
0811 00:31:32.281499 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:32.281505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:32.281624 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:32.281761 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:32.283075 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.283888 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.285765 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.286235 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.286412 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:32.288105 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:32.288111 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:32.288115 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:32.288150 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:32.288200 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:32.288214 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:32.288326 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:32.288660 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 00:31:32.288790 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":779,"thread_start_us":620,"threads_started":1}
I20260811 00:31:32.290463 24351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:32.277691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:39989 (local address 127.2.120.2:43873)
0811 00:31:32.279035 (+ 1344us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:32.279099 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:32.281159 (+ 2060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:32.282065 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:32.282075 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:32.282197 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:32.282868 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:32.282885 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.284072 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.284083 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:32.285578 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:32.285594 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.286546 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.286554 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:32.286821 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:32.287682 (+ 861us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:31:32.287686 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:32.287734 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:31:32.287833 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:32.288328 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:32.288344 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:32.288355 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 00:31:32.288650 (+ 295us) client_negotiation.cc:240] Negotiation successful
0811 00:31:32.289154 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":451,"threads_started":1}
I20260811 00:31:32.292569 24348 heartbeater.cc:347] Connected to a master server at 127.2.120.62:39989
I20260811 00:31:32.293004 24348 heartbeater.cc:464] Registering TS with master...
I20260811 00:31:32.294196 24348 heartbeater.cc:511] Master 127.2.120.62:39989 requested a full tablet report, sending...
I20260811 00:31:32.297008 23977 ts_manager.cc:194] Registered new tserver with Master: 8e7bb54c9dad48a1a1f30e45c5f1cd0c (127.2.120.2:37423)
I20260811 00:31:32.298403 23977 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.120.2:43873
I20260811 00:31:32.694479 24198 heartbeater.cc:503] Master 127.2.120.62:39989 was elected leader, sending a full tablet report...
W20260811 00:31:32.712203 24352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:32.712725 24352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:32.712893 24352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:32.725629 24352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:32.725832 24352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:32.726037 24352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:31:32.739532 24352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:35351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:39989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:32.741291 24352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:32.743553 24352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:32.761214 24359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:32.761492 24361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:32.761358 24358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:32.763377 24352 server_base.cc:1034] running on GCE node
I20260811 00:31:32.764287 24352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:32.765750 24352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:32.767063 24352 hybrid_clock.cc:648] HybridClock initialized: now 1786408292766996 us; error 60 us; skew 500 ppm
I20260811 00:31:32.767544 24352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:31:32.770749 24352 webserver.cc:460] Webserver started at http://127.2.120.3:36441/ using document root <none> and password file <none>
I20260811 00:31:32.772653 24352 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:32.773058 24352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:32.773674 24352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:32.777383 24352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "0807b01666ff449786a61d201782d7c2"
format_stamp: "Formatted at 2026-08-11 00:31:32 on dist-test-slave-fd7l"
server_key: "2d606248b86a58acad0cf05ee893eddd"
server_key_iv: "1e2fe0e891725f53f2b9c2ceaa4f9f41"
server_key_version: "encryptionkey@0"
I20260811 00:31:32.778352 24352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "0807b01666ff449786a61d201782d7c2"
format_stamp: "Formatted at 2026-08-11 00:31:32 on dist-test-slave-fd7l"
server_key: "2d606248b86a58acad0cf05ee893eddd"
server_key_iv: "1e2fe0e891725f53f2b9c2ceaa4f9f41"
server_key_version: "encryptionkey@0"
I20260811 00:31:32.785503 24352 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 00:31:32.790915 24367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:32.792774 24352 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:31:32.793032 24352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "0807b01666ff449786a61d201782d7c2"
format_stamp: "Formatted at 2026-08-11 00:31:32 on dist-test-slave-fd7l"
server_key: "2d606248b86a58acad0cf05ee893eddd"
server_key_iv: "1e2fe0e891725f53f2b9c2ceaa4f9f41"
server_key_version: "encryptionkey@0"
I20260811 00:31:32.793341 24352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:32.812448 24352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:32.813930 24352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:32.814329 24352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:32.816115 24352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:32.819056 24352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:32.819233 24352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:32.819367 24352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:32.819456 24352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:32.865136 24352 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:37037
I20260811 00:31:32.865326 24494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:37037 every 8 connection(s)
I20260811 00:31:32.866889 24352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:31:32.877223 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 24352
I20260811 00:31:32.877480 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:31:32.878139 2528 external_mini_cluster.cc:1428] Setting key 074a4862924072868726da74c2b9c7f7
I20260811 00:31:32.884829 24497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:32.870272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:48425 (local address 127.2.120.62:39989)
0811 00:31:32.870841 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:32.870846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:32.872679 (+ 1833us) server_negotiation.cc:407] Connection header received
0811 00:31:32.874859 (+ 2180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:32.874865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:32.874951 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:32.875078 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:32.876675 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.877675 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.880467 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.881057 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.881249 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:32.883367 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:32.883373 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:32.883376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:32.883399 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:32.883451 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:32.883465 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:32.883588 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:32.884119 (+ 531us) server_negotiation.cc:299] Negotiation successful
0811 00:31:32.884298 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":302,"threads_started":1}
I20260811 00:31:32.886489 24498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:32.870990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:39989 (local address 127.2.120.3:48425)
0811 00:31:32.872400 (+ 1410us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:32.872466 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:32.874478 (+ 2012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:32.875390 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:32.875401 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:32.875545 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:32.876355 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:32.876378 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.877880 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.877887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:32.880224 (+ 2337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:32.880247 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.881392 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.881398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:32.881675 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:32.882782 (+ 1107us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:31:32.882787 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:32.882878 (+ 91us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:31:32.883037 (+ 159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:32.883654 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:32.883666 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:32.883676 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 00:31:32.884029 (+ 353us) client_negotiation.cc:240] Negotiation successful
0811 00:31:32.884684 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":528,"threads_started":1}
I20260811 00:31:32.889071 24495 heartbeater.cc:347] Connected to a master server at 127.2.120.62:39989
I20260811 00:31:32.889576 24495 heartbeater.cc:464] Registering TS with master...
I20260811 00:31:32.890864 24495 heartbeater.cc:511] Master 127.2.120.62:39989 requested a full tablet report, sending...
I20260811 00:31:32.893021 23977 ts_manager.cc:194] Registered new tserver with Master: 0807b01666ff449786a61d201782d7c2 (127.2.120.3:37037)
I20260811 00:31:32.894218 23977 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.120.3:48425
I20260811 00:31:32.895934 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:31:32.910192 24497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:32.901489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57180 (local address 127.2.120.62:39989)
0811 00:31:32.901720 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:32.901727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:32.902338 (+ 611us) server_negotiation.cc:407] Connection header received
0811 00:31:32.902508 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:32.902514 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:32.902647 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:32.902771 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:32.903452 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.904678 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.906283 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.907329 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.907620 (+ 291us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:32.909154 (+ 1534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:32.909162 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:32.909167 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:32.909198 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:32.909246 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:32.909263 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:32.909471 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:32.909707 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 00:31:32.909784 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 00:31:32.916468 23977 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:31:32.925266 23977 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:31:32.955701 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:32.940861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:37423 (local address 127.0.0.1:48452)
0811 00:31:32.941789 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:32.941838 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:32.942098 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:32.943768 (+ 1670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:32.943776 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:32.943831 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:32.944406 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:32.944427 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.947709 (+ 3282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.947715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:32.950877 (+ 3162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:32.950896 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.954164 (+ 3268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.954171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:32.954577 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:32.954767 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 00:31:32.954922 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 00:31:32.955202 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":298,"thread_start_us":390,"threads_started":1}
I20260811 00:31:32.955755 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:32.941132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:37037 (local address 127.0.0.1:47248)
0811 00:31:32.942779 (+ 1647us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:32.942806 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:32.942991 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:32.946183 (+ 3192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:32.946188 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:32.946214 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:32.946706 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:32.946719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.948658 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.948666 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:32.951312 (+ 2646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:32.951327 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.954645 (+ 3318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.954654 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:32.954854 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:32.954904 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 00:31:32.954991 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 00:31:32.955202 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1418,"mutex_wait_us":28,"thread_start_us":2454,"threads_started":1}
I20260811 00:31:32.955945 24515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:32.944048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44118 (local address 127.2.120.1:37999)
0811 00:31:32.945218 (+ 1170us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:32.945231 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:32.945268 (+ 37us) server_negotiation.cc:407] Connection header received
0811 00:31:32.945382 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:32.945391 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:32.945692 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:32.945896 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:32.946722 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.947892 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.950808 (+ 2916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.953294 (+ 2486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.953511 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:32.953854 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:32.955140 (+ 1286us) server_negotiation.cc:299] Negotiation successful
0811 00:31:32.955431 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":983,"thread_start_us":815,"threads_started":1}
I20260811 00:31:32.956681 24513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:32.942233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47248 (local address 127.2.120.3:37037)
0811 00:31:32.945604 (+ 3371us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:32.945612 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:32.945644 (+ 32us) server_negotiation.cc:407] Connection header received
0811 00:31:32.945739 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:32.945747 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:32.945975 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:32.946182 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:32.946914 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.948403 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.951623 (+ 3220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.954244 (+ 2621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.954534 (+ 290us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:32.954882 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:32.955138 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 00:31:32.955431 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3203,"thread_start_us":3428,"threads_started":1}
I20260811 00:31:32.956972 24514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:32.942908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:37999 (local address 127.0.0.1:44118)
0811 00:31:32.944892 (+ 1984us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:32.944918 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:32.945052 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:32.945925 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:32.945929 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:32.945958 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:32.946454 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:32.946480 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.948075 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.948080 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:32.950379 (+ 2299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:32.950395 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.954162 (+ 3767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.954172 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:32.954661 (+ 489us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:32.954771 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 00:31:32.954957 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 00:31:32.956441 (+ 1484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1814,"mutex_wait_us":1347,"thread_start_us":1647,"threads_started":1}
I20260811 00:31:32.958366 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:32.941682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48452 (local address 127.2.120.2:37423)
0811 00:31:32.942719 (+ 1037us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:32.942729 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:32.942771 (+ 42us) server_negotiation.cc:407] Connection header received
0811 00:31:32.942920 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:32.942932 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:32.943395 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:32.945600 (+ 2205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:32.945867 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.947520 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.951168 (+ 3648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:32.953903 (+ 2735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:32.954093 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:32.954349 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:32.956385 (+ 2036us) server_negotiation.cc:299] Negotiation successful
0811 00:31:32.956651 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":878,"thread_start_us":857,"threads_started":1}
I20260811 00:31:32.961789 24124 tablet_service.cc:1467] Processing CreateTablet for tablet a2023f43604342df8e5ac5f0e8f632ba (DEFAULT_TABLE table=test-table [id=5c1695ddbb2745d8aba1c527917e3c82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:31:32.961790 24419 tablet_service.cc:1467] Processing CreateTablet for tablet a2023f43604342df8e5ac5f0e8f632ba (DEFAULT_TABLE table=test-table [id=5c1695ddbb2745d8aba1c527917e3c82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:31:32.963801 24419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2023f43604342df8e5ac5f0e8f632ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:32.963801 24124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2023f43604342df8e5ac5f0e8f632ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:32.965003 24275 tablet_service.cc:1467] Processing CreateTablet for tablet a2023f43604342df8e5ac5f0e8f632ba (DEFAULT_TABLE table=test-table [id=5c1695ddbb2745d8aba1c527917e3c82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:31:32.966706 24275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2023f43604342df8e5ac5f0e8f632ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:32.981473 24516 tablet_bootstrap.cc:492] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c: Bootstrap starting.
I20260811 00:31:32.982445 24518 tablet_bootstrap.cc:492] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2: Bootstrap starting.
I20260811 00:31:32.983007 24517 tablet_bootstrap.cc:492] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22: Bootstrap starting.
I20260811 00:31:32.985860 24516 tablet_bootstrap.cc:654] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:32.986317 24518 tablet_bootstrap.cc:654] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:32.986708 24517 tablet_bootstrap.cc:654] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:32.987337 24516 log.cc:824] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:32.987437 24518 log.cc:824] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:32.987968 24517 log.cc:824] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:32.991408 24518 tablet_bootstrap.cc:492] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2: No bootstrap required, opened a new log
I20260811 00:31:32.991496 24516 tablet_bootstrap.cc:492] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c: No bootstrap required, opened a new log
I20260811 00:31:32.991766 24518 ts_tablet_manager.cc:1397] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260811 00:31:32.991792 24516 ts_tablet_manager.cc:1397] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260811 00:31:32.992944 24517 tablet_bootstrap.cc:492] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22: No bootstrap required, opened a new log
I20260811 00:31:32.993274 24517 ts_tablet_manager.cc:1397] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260811 00:31:33.000553 24518 raft_consensus.cc:348] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 37423 } } peers { permanent_uuid: "0807b01666ff449786a61d201782d7c2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37037 } } peers { permanent_uuid: "26cb7439e4bb4593a63a551577f1ba22" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37999 } }
I20260811 00:31:33.001327 24518 raft_consensus.cc:374] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:33.001449 24518 raft_consensus.cc:729] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0807b01666ff449786a61d201782d7c2, State: Initialized, Role: FOLLOWER
I20260811 00:31:33.002632 24518 consensus_queue.cc:260] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 37423 } } peers { permanent_uuid: "0807b01666ff449786a61d201782d7c2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37037 } } peers { permanent_uuid: "26cb7439e4bb4593a63a551577f1ba22" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37999 } }
I20260811 00:31:33.003505 24516 raft_consensus.cc:348] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 37423 } } peers { permanent_uuid: "0807b01666ff449786a61d201782d7c2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37037 } } peers { permanent_uuid: "26cb7439e4bb4593a63a551577f1ba22" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37999 } }
I20260811 00:31:33.004081 24516 raft_consensus.cc:374] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:33.004200 24516 raft_consensus.cc:729] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e7bb54c9dad48a1a1f30e45c5f1cd0c, State: Initialized, Role: FOLLOWER
I20260811 00:31:33.004976 24495 heartbeater.cc:503] Master 127.2.120.62:39989 was elected leader, sending a full tablet report...
I20260811 00:31:33.004812 24517 raft_consensus.cc:348] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 37423 } } peers { permanent_uuid: "0807b01666ff449786a61d201782d7c2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37037 } } peers { permanent_uuid: "26cb7439e4bb4593a63a551577f1ba22" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37999 } }
I20260811 00:31:33.005275 24517 raft_consensus.cc:374] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:33.005108 24516 consensus_queue.cc:260] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 37423 } } peers { permanent_uuid: "0807b01666ff449786a61d201782d7c2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37037 } } peers { permanent_uuid: "26cb7439e4bb4593a63a551577f1ba22" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37999 } }
I20260811 00:31:33.005373 24517 raft_consensus.cc:729] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26cb7439e4bb4593a63a551577f1ba22, State: Initialized, Role: FOLLOWER
I20260811 00:31:33.005884 24518 ts_tablet_manager.cc:1428] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260811 00:31:33.006052 24517 consensus_queue.cc:260] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 37423 } } peers { permanent_uuid: "0807b01666ff449786a61d201782d7c2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37037 } } peers { permanent_uuid: "26cb7439e4bb4593a63a551577f1ba22" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37999 } }
I20260811 00:31:33.008152 24348 heartbeater.cc:503] Master 127.2.120.62:39989 was elected leader, sending a full tablet report...
I20260811 00:31:33.008392 24516 ts_tablet_manager.cc:1428] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260811 00:31:33.008577 24517 ts_tablet_manager.cc:1428] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
W20260811 00:31:33.025095 24349 tablet.cc:2369] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:31:33.119151 24496 tablet.cc:2369] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:31:33.161011 24199 tablet.cc:2369] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:31:33.337514 24523 raft_consensus.cc:482] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:31:33.337903 24523 raft_consensus.cc:504] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 37423 } } peers { permanent_uuid: "0807b01666ff449786a61d201782d7c2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37037 } } peers { permanent_uuid: "26cb7439e4bb4593a63a551577f1ba22" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37999 } }
I20260811 00:31:33.339910 24523 leader_election.cc:288] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0807b01666ff449786a61d201782d7c2 (127.2.120.3:37037), 26cb7439e4bb4593a63a551577f1ba22 (127.2.120.1:37999)
I20260811 00:31:33.343024 24522 raft_consensus.cc:482] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:31:33.343394 24522 raft_consensus.cc:504] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 37423 } } peers { permanent_uuid: "0807b01666ff449786a61d201782d7c2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37037 } } peers { permanent_uuid: "26cb7439e4bb4593a63a551577f1ba22" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37999 } }
I20260811 00:31:33.345755 24522 leader_election.cc:288] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e7bb54c9dad48a1a1f30e45c5f1cd0c (127.2.120.2:37423), 26cb7439e4bb4593a63a551577f1ba22 (127.2.120.1:37999)
I20260811 00:31:33.350366 24526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.340469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:37999 (local address 127.2.120.2:52981)
0811 00:31:33.341864 (+ 1395us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:33.341880 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:33.342054 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:33.342456 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:33.342461 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:33.342495 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:33.342916 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:33.342928 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.344135 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.344141 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:33.347467 (+ 3326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:33.347481 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.349477 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.349483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:33.349662 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:33.349705 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 00:31:33.349812 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 00:31:33.349987 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1269,"mutex_wait_us":108,"thread_start_us":488,"threads_started":1}
I20260811 00:31:33.350435 24513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.340451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:60979 (local address 127.2.120.3:37037)
0811 00:31:33.340711 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:33.340716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:33.341480 (+ 764us) server_negotiation.cc:407] Connection header received
0811 00:31:33.341733 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:33.341740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:33.341816 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:33.341924 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:33.342511 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.343627 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.346688 (+ 3061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.348932 (+ 2244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.349071 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:33.349197 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:33.350025 (+ 828us) server_negotiation.cc:299] Negotiation successful
0811 00:31:33.350185 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260811 00:31:33.351127 24515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.340452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:52981 (local address 127.2.120.1:37999)
0811 00:31:33.340888 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:33.340895 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:33.341998 (+ 1103us) server_negotiation.cc:407] Connection header received
0811 00:31:33.342244 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:33.342248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:33.342329 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:33.342440 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:33.343095 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.343991 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.347707 (+ 3716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.349104 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.349230 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:33.349354 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:33.350679 (+ 1325us) server_negotiation.cc:299] Negotiation successful
0811 00:31:33.350816 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":282}
I20260811 00:31:33.351120 24525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.340448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:37037 (local address 127.2.120.2:60979)
0811 00:31:33.341340 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:33.341358 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:33.341572 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:33.341949 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:33.341953 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:33.341980 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:33.342358 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:33.342367 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.343841 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.343846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:33.346413 (+ 2567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:33.346432 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.349340 (+ 2908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.349347 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:33.349542 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:33.349854 (+ 312us) client_negotiation.cc:769] Sending connection context
0811 00:31:33.350606 (+ 752us) client_negotiation.cc:240] Negotiation successful
0811 00:31:33.350759 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":368,"thread_start_us":1133,"threads_started":1}
I20260811 00:31:33.353516 24147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2023f43604342df8e5ac5f0e8f632ba" candidate_uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26cb7439e4bb4593a63a551577f1ba22" is_pre_election: true
I20260811 00:31:33.353518 24445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2023f43604342df8e5ac5f0e8f632ba" candidate_uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0807b01666ff449786a61d201782d7c2" is_pre_election: true
I20260811 00:31:33.354091 24445 raft_consensus.cc:2454] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e7bb54c9dad48a1a1f30e45c5f1cd0c in term 0.
I20260811 00:31:33.354147 24147 raft_consensus.cc:2454] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e7bb54c9dad48a1a1f30e45c5f1cd0c in term 0.
I20260811 00:31:33.355007 24498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.346025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:37423 (local address 127.2.120.3:38965)
0811 00:31:33.346196 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:33.346221 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:33.346364 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:33.347123 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:33.347130 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:33.347168 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:33.348280 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:33.348297 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.349926 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.349932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:33.352221 (+ 2289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:33.352245 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.354102 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.354108 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:33.354286 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:33.354320 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 00:31:33.354413 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 00:31:33.354581 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 00:31:33.354983 24221 leader_election.cc:302] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26cb7439e4bb4593a63a551577f1ba22, 8e7bb54c9dad48a1a1f30e45c5f1cd0c; no voters:
I20260811 00:31:33.355438 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.346472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:38965 (local address 127.2.120.2:37423)
0811 00:31:33.346664 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:33.346670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:33.346689 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:31:33.346751 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:33.346755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:33.346841 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:33.346930 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:33.348467 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.349639 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.352405 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.353691 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.353909 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:33.354014 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:33.354947 (+ 933us) server_negotiation.cc:299] Negotiation successful
0811 00:31:33.355131 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":48}
I20260811 00:31:33.355513 24523 raft_consensus.cc:2781] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:31:33.355685 24523 raft_consensus.cc:482] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:31:33.355837 24523 raft_consensus.cc:3037] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:33.356820 24299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2023f43604342df8e5ac5f0e8f632ba" candidate_uuid: "0807b01666ff449786a61d201782d7c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c" is_pre_election: true
I20260811 00:31:33.358493 24527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.346420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:37999 (local address 127.2.120.3:55043)
0811 00:31:33.348776 (+ 2356us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:33.348798 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:33.348919 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:33.349349 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:33.349355 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:33.349380 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:33.349834 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:33.349855 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.352253 (+ 2398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.352257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:33.354622 (+ 2365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:33.354638 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.357894 (+ 3256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.357899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:33.358027 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:33.358055 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 00:31:33.358142 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 00:31:33.358246 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2212,"thread_start_us":861,"threads_started":1}
I20260811 00:31:33.358822 24528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.347009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:55043 (local address 127.2.120.1:37999)
0811 00:31:33.348214 (+ 1205us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:33.348222 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:33.349034 (+ 812us) server_negotiation.cc:407] Connection header received
0811 00:31:33.349103 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:33.349109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:33.349185 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:33.349270 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:33.350155 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.352033 (+ 1878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.355410 (+ 3377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.357723 (+ 2313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.357862 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:33.357992 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:33.358196 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 00:31:33.358346 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":297,"server-negotiator.queue_time_us":1033,"thread_start_us":784,"threads_started":1}
I20260811 00:31:33.359269 24523 raft_consensus.cc:504] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 37423 } } peers { permanent_uuid: "0807b01666ff449786a61d201782d7c2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37037 } } peers { permanent_uuid: "26cb7439e4bb4593a63a551577f1ba22" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37999 } }
I20260811 00:31:33.359459 24147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2023f43604342df8e5ac5f0e8f632ba" candidate_uuid: "0807b01666ff449786a61d201782d7c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26cb7439e4bb4593a63a551577f1ba22" is_pre_election: true
I20260811 00:31:33.359733 24147 raft_consensus.cc:2454] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0807b01666ff449786a61d201782d7c2 in term 0.
I20260811 00:31:33.359799 24299 raft_consensus.cc:2379] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0807b01666ff449786a61d201782d7c2 in current term 1: Already voted for candidate 8e7bb54c9dad48a1a1f30e45c5f1cd0c in this term.
I20260811 00:31:33.360337 24369 leader_election.cc:302] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0807b01666ff449786a61d201782d7c2, 26cb7439e4bb4593a63a551577f1ba22; no voters:
I20260811 00:31:33.360538 24523 leader_election.cc:288] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [CANDIDATE]: Term 1 election: Requested vote from peers 0807b01666ff449786a61d201782d7c2 (127.2.120.3:37037), 26cb7439e4bb4593a63a551577f1ba22 (127.2.120.1:37999)
I20260811 00:31:33.360656 24445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2023f43604342df8e5ac5f0e8f632ba" candidate_uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0807b01666ff449786a61d201782d7c2"
I20260811 00:31:33.360869 24445 raft_consensus.cc:3037] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:33.363580 24445 raft_consensus.cc:2454] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e7bb54c9dad48a1a1f30e45c5f1cd0c in term 1.
I20260811 00:31:33.363858 24522 raft_consensus.cc:2741] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260811 00:31:33.364063 24147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2023f43604342df8e5ac5f0e8f632ba" candidate_uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26cb7439e4bb4593a63a551577f1ba22"
I20260811 00:31:33.364107 24223 leader_election.cc:302] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0807b01666ff449786a61d201782d7c2, 8e7bb54c9dad48a1a1f30e45c5f1cd0c; no voters:
I20260811 00:31:33.364337 24147 raft_consensus.cc:3037] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:33.364494 24523 raft_consensus.cc:2781] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:33.365141 24523 raft_consensus.cc:686] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [term 1 LEADER]: Becoming Leader. State: Replica: 8e7bb54c9dad48a1a1f30e45c5f1cd0c, State: Running, Role: LEADER
I20260811 00:31:33.365792 24523 consensus_queue.cc:237] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 37423 } } peers { permanent_uuid: "0807b01666ff449786a61d201782d7c2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37037 } } peers { permanent_uuid: "26cb7439e4bb4593a63a551577f1ba22" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37999 } }
I20260811 00:31:33.367262 24147 raft_consensus.cc:2454] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e7bb54c9dad48a1a1f30e45c5f1cd0c in term 1.
I20260811 00:31:33.371613 23977 catalog_manager.cc:5495] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e7bb54c9dad48a1a1f30e45c5f1cd0c (127.2.120.2). New cstate: current_term: 1 leader_uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e7bb54c9dad48a1a1f30e45c5f1cd0c" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 37423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0807b01666ff449786a61d201782d7c2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26cb7439e4bb4593a63a551577f1ba22" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37999 } health_report { overall_health: UNKNOWN } } }
I20260811 00:31:33.433156 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.424313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55150 (local address 127.2.120.2:37423)
0811 00:31:33.424660 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:33.424670 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:33.425102 (+ 432us) server_negotiation.cc:407] Connection header received
0811 00:31:33.425304 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:33.425309 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:33.425430 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:33.425583 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:31:33.426135 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.427012 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.429067 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.429720 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.429899 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:33.430649 (+ 750us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:31:33.431744 (+ 1095us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:31:33.431975 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:33.432699 (+ 724us) server_negotiation.cc:299] Negotiation successful
0811 00:31:33.432768 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 00:31:33.445163 24445 raft_consensus.cc:1261] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2 [term 1 FOLLOWER]: Refusing update from remote peer 8e7bb54c9dad48a1a1f30e45c5f1cd0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:31:33.445187 24147 raft_consensus.cc:1261] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22 [term 1 FOLLOWER]: Refusing update from remote peer 8e7bb54c9dad48a1a1f30e45c5f1cd0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:31:33.446565 24523 consensus_queue.cc:1035] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "26cb7439e4bb4593a63a551577f1ba22" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 37999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:31:33.447247 24529 consensus_queue.cc:1035] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0807b01666ff449786a61d201782d7c2" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 37037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:31:33.494663 23977 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:31:33.494975 23977 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:31:33.499363 23977 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e690188a5714aa5a51419447cd85e65: Sending DeleteTablet for 3 replicas of tablet a2023f43604342df8e5ac5f0e8f632ba
I20260811 00:31:33.500871 24124 tablet_service.cc:1514] Processing DeleteTablet for tablet a2023f43604342df8e5ac5f0e8f632ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:31:33 UTC) from {username='slave'} at 127.0.0.1:44118
I20260811 00:31:33.501008 24419 tablet_service.cc:1514] Processing DeleteTablet for tablet a2023f43604342df8e5ac5f0e8f632ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:31:33 UTC) from {username='slave'} at 127.0.0.1:47248
I20260811 00:31:33.501073 24275 tablet_service.cc:1514] Processing DeleteTablet for tablet a2023f43604342df8e5ac5f0e8f632ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:31:33 UTC) from {username='slave'} at 127.0.0.1:48452
I20260811 00:31:33.503194 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 24045
I20260811 00:31:33.516856 24547 tablet_replica.cc:330] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c: stopping tablet replica
I20260811 00:31:33.517503 24547 raft_consensus.cc:2229] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:31:33.518396 24547 raft_consensus.cc:2258] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:31:33.520934 24547 ts_tablet_manager.cc:1905] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:31:33.529668 24547 ts_tablet_manager.cc:1918] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:31:33.529940 24547 log.cc:1196] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/a2023f43604342df8e5ac5f0e8f632ba
I20260811 00:31:33.530644 24547 ts_tablet_manager.cc:1939] T a2023f43604342df8e5ac5f0e8f632ba P 8e7bb54c9dad48a1a1f30e45c5f1cd0c: Deleting consensus metadata
I20260811 00:31:33.533722 23957 catalog_manager.cc:4841] TS 8e7bb54c9dad48a1a1f30e45c5f1cd0c (127.2.120.2:37423): tablet a2023f43604342df8e5ac5f0e8f632ba (table test-table [id=5c1695ddbb2745d8aba1c527917e3c82]) successfully deleted
I20260811 00:31:33.536056 24545 tablet_replica.cc:330] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2: stopping tablet replica
I20260811 00:31:33.536684 24546 tablet_replica.cc:330] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22: stopping tablet replica
I20260811 00:31:33.536940 24545 raft_consensus.cc:2229] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:31:33.537575 24545 raft_consensus.cc:2258] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:31:33.537184 24546 raft_consensus.cc:2229] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:31:33.538489 24546 raft_consensus.cc:2258] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:31:33.539888 24545 ts_tablet_manager.cc:1905] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:31:33.540946 24546 ts_tablet_manager.cc:1905] T a2023f43604342df8e5ac5f0e8f632ba P 26cb7439e4bb4593a63a551577f1ba22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:31:33.543735 24528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.505178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60248 (local address 127.2.120.1:37999)
0811 00:31:33.505385 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:33.505392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:33.531278 (+ 25886us) server_negotiation.cc:407] Connection header received
0811 00:31:33.531656 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:33.531665 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:33.531832 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:33.531956 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:33.533479 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.535250 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.538645 (+ 3395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.539634 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.539776 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:33.541759 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:33.541776 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:33.541795 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:33.542681 (+ 886us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:33.542757 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:33.542788 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:33.542916 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:33.543208 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 00:31:33.543351 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:31:33.545277 24191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:33.548840 24545 ts_tablet_manager.cc:1918] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:31:33.549410 24545 log.cc:1196] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/a2023f43604342df8e5ac5f0e8f632ba
I20260811 00:31:33.550375 24545 ts_tablet_manager.cc:1939] T a2023f43604342df8e5ac5f0e8f632ba P 0807b01666ff449786a61d201782d7c2: Deleting consensus metadata
I20260811 00:31:33.553478 23961 catalog_manager.cc:4841] TS 0807b01666ff449786a61d201782d7c2 (127.2.120.3:37037): tablet a2023f43604342df8e5ac5f0e8f632ba (table test-table [id=5c1695ddbb2745d8aba1c527917e3c82]) successfully deleted
I20260811 00:31:33.650092 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 24045
W20260811 00:31:33.680924 23959 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.120.1:37999 (error 108)
I20260811 00:31:33.681358 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 24202
I20260811 00:31:33.683766 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.682804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:37999 (local address 127.0.0.1:60258)
0811 00:31:33.683359 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:33.683510 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:37999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":280,"threads_started":1}
W20260811 00:31:33.684111 23959 catalog_manager.cc:4584] TS 26cb7439e4bb4593a63a551577f1ba22 (127.2.120.1:37999): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a2023f43604342df8e5ac5f0e8f632ba: Network error: Client connection negotiation failed: client connection to 127.2.120.1:37999: connect: Connection refused (error 111)
I20260811 00:31:33.690342 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.682315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55158 (local address 127.2.120.2:37423)
0811 00:31:33.682519 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:33.682526 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:33.682549 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:31:33.682757 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:33.682764 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:33.682891 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:33.683021 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:33.683725 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.684950 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.686343 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.687046 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.687191 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:33.688408 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:33.688420 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:33.688442 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:33.689402 (+ 960us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:33.689471 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:33.689500 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:33.689632 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:33.689823 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 00:31:33.689944 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 00:31:33.691258 24343 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:33.738011 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.737378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:37999 (local address 127.0.0.1:60266)
0811 00:31:33.737691 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:33.737820 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:37999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260811 00:31:33.739835 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.738948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:37999 (local address 127.0.0.1:60280)
0811 00:31:33.739345 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:33.739518 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:37999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260811 00:31:33.801285 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.800409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:37999 (local address 127.0.0.1:60282)
0811 00:31:33.800813 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:33.800991 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:37999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260811 00:31:33.802963 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.802340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:37999 (local address 127.0.0.1:60292)
0811 00:31:33.802608 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:33.802727 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:37999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260811 00:31:33.835837 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 24202
I20260811 00:31:33.866981 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 24352
I20260811 00:31:33.878108 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.868031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32974 (local address 127.2.120.3:37037)
0811 00:31:33.868679 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:33.868684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:33.868703 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:31:33.868764 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:33.868768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:33.868871 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:33.868984 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:33.869557 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.870438 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.873098 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:33.873645 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:33.873767 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:33.875664 (+ 1897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:33.875688 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:33.875718 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:33.876812 (+ 1094us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:33.876910 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:33.876942 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:33.877115 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:33.877366 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 00:31:33.877515 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":347,"threads_started":1}
I20260811 00:31:33.879158 24490 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:33.884855 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.883770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:37999 (local address 127.0.0.1:60296)
0811 00:31:33.884254 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:33.884415 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:37999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276}
I20260811 00:31:33.886749 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:33.886034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:37999 (local address 127.0.0.1:60300)
0811 00:31:33.886396 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:33.886515 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:37999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233}
I20260811 00:31:33.991385 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 24352
I20260811 00:31:34.021870 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 23940
I20260811 00:31:34.023226 24012 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:34.154460 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:34.153827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:37999 (local address 127.0.0.1:60316)
0811 00:31:34.154081 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:34.154247 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:37999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 00:31:34.154666 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 23940
2026-08-11T00:31:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3945 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:31:34 dist-test-slave-fd7l krb5kdc[24560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:31:34 dist-test-slave-fd7l krb5kdc[24560](info): set up 2 sockets
Aug 11 00:31:34 dist-test-slave-fd7l krb5kdc[24560](info): commencing operation
krb5kdc: starting...
W20260811 00:31:35.443207 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.001s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:31:35 dist-test-slave-fd7l krb5kdc[24560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:31:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:31:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:31:35.786221 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:45801
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:46025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:45801
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:31:36.175995 24576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:36.176407 24576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:36.176604 24576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:36.184283 24576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:31:36.184482 24576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:36.184633 24576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:36.184741 24576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:31:36.184831 24576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:31:36.195804 24576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:46025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:45801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:45801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:36.197479 24576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:36.198994 24576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:36.214229 24581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:36.214154 24584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:36.214202 24582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:36.215111 24576 server_base.cc:1034] running on GCE node
I20260811 00:31:36.216118 24576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:36.217957 24576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:36.219234 24576 hybrid_clock.cc:648] HybridClock initialized: now 1786408296219172 us; error 58 us; skew 500 ppm
I20260811 00:31:36.219669 24576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:36 dist-test-slave-fd7l krb5kdc[24560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408296, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:36.223616 24576 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:31:36.225783 24576 webserver.cc:460] Webserver started at http://127.2.120.62:46237/ using document root <none> and password file <none>
I20260811 00:31:36.226886 24576 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:36.227089 24576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:36.227450 24576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:36.231006 24576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "b82e7457a935450db3369c7a35cae3dd"
format_stamp: "Formatted at 2026-08-11 00:31:36 on dist-test-slave-fd7l"
server_key: "4ee1398960f2d7a7902094b42ce8003e"
server_key_iv: "8202c996df591edd2aeb910b6d10570e"
server_key_version: "encryptionkey@0"
I20260811 00:31:36.231909 24576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "b82e7457a935450db3369c7a35cae3dd"
format_stamp: "Formatted at 2026-08-11 00:31:36 on dist-test-slave-fd7l"
server_key: "4ee1398960f2d7a7902094b42ce8003e"
server_key_iv: "8202c996df591edd2aeb910b6d10570e"
server_key_version: "encryptionkey@0"
I20260811 00:31:36.238083 24576 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 00:31:36.242167 24593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:36.243690 24576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:31:36.243924 24576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "b82e7457a935450db3369c7a35cae3dd"
format_stamp: "Formatted at 2026-08-11 00:31:36 on dist-test-slave-fd7l"
server_key: "4ee1398960f2d7a7902094b42ce8003e"
server_key_iv: "8202c996df591edd2aeb910b6d10570e"
server_key_version: "encryptionkey@0"
I20260811 00:31:36.244175 24576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:36.265084 24576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:36.270505 24576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:36.270864 24576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:36.292963 24576 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:45801
I20260811 00:31:36.292984 24653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:45801 every 8 connection(s)
I20260811 00:31:36.294538 24576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:31:36.298699 24654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:36.302881 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 24576
I20260811 00:31:36.303104 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:31:36.303682 2528 external_mini_cluster.cc:1428] Setting key 64cb13a34ad8fd8dba0abe9e06c22a14
I20260811 00:31:36.313136 24654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd: Bootstrap starting.
I20260811 00:31:36.317430 24654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:36.318921 24654 log.cc:824] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:31:36 dist-test-slave-fd7l krb5kdc[24560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:31:36.322494 24654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd: No bootstrap required, opened a new log
I20260811 00:31:36.331365 24658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:36.305996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58076 (local address 127.2.120.62:45801)
0811 00:31:36.306916 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:36.306929 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:36.306974 (+ 45us) server_negotiation.cc:407] Connection header received
0811 00:31:36.308790 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:36.308814 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:36.309235 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:36.309591 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:36.311432 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:36.312967 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:36.314900 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:36.315664 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:36.315826 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:36.322208 (+ 6382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:36.322223 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:36.322250 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:36.322313 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:36.326342 (+ 4029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:36.327362 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:36.327371 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:36.327382 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:36.327595 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:36.328041 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:36.328053 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:36.328060 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:36.328820 (+ 760us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:36.329197 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:36.329689 (+ 492us) server_negotiation.cc:299] Negotiation successful
0811 00:31:36.330217 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":448,"threads_started":1}
I20260811 00:31:36.331235 24654 raft_consensus.cc:348] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82e7457a935450db3369c7a35cae3dd" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 45801 } }
I20260811 00:31:36.331773 24654 raft_consensus.cc:374] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:36.331880 24654 raft_consensus.cc:729] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b82e7457a935450db3369c7a35cae3dd, State: Initialized, Role: FOLLOWER
I20260811 00:31:36.332581 24654 consensus_queue.cc:260] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82e7457a935450db3369c7a35cae3dd" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 45801 } }
I20260811 00:31:36.332865 24654 raft_consensus.cc:388] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:31:36.333024 24654 raft_consensus.cc:482] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:31:36.333223 24654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:36.335902 24654 raft_consensus.cc:504] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82e7457a935450db3369c7a35cae3dd" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 45801 } }
I20260811 00:31:36.336613 24654 leader_election.cc:302] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b82e7457a935450db3369c7a35cae3dd; no voters:
I20260811 00:31:36.337347 24654 leader_election.cc:288] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:31:36.337539 24660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:36.338424 24660 raft_consensus.cc:686] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd [term 1 LEADER]: Becoming Leader. State: Replica: b82e7457a935450db3369c7a35cae3dd, State: Running, Role: LEADER
I20260811 00:31:36.339005 24660 consensus_queue.cc:237] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82e7457a935450db3369c7a35cae3dd" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 45801 } }
I20260811 00:31:36.339708 24654 sys_catalog.cc:564] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:31:36.342242 24661 sys_catalog.cc:455] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b82e7457a935450db3369c7a35cae3dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82e7457a935450db3369c7a35cae3dd" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 45801 } } }
I20260811 00:31:36.342175 24662 sys_catalog.cc:455] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader b82e7457a935450db3369c7a35cae3dd. Latest consensus state: current_term: 1 leader_uuid: "b82e7457a935450db3369c7a35cae3dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82e7457a935450db3369c7a35cae3dd" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 45801 } } }
I20260811 00:31:36.342638 24661 sys_catalog.cc:458] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:36.342705 24662 sys_catalog.cc:458] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:36.343662 24667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:31:36.351718 24667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:31:36.363018 24667 catalog_manager.cc:1347] Generated new cluster ID: 46496c4779da4ef49cb3e1ac29d3bb55
I20260811 00:31:36.363199 24667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:31:36.379297 24667 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:31:36.381439 24667 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:31:36.391531 24667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd: Generated new TSK 0
I20260811 00:31:36.392858 24667 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.120.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
Entry for principal kudu/127.2.120.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab.
I20260811 00:31:36.484206 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:45801
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:46025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:31:36.895073 24684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:36.895383 24684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:36.895546 24684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:36.903012 24684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:36.903127 24684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:36.903347 24684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:31:36.914449 24684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:46025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.1.keytab
--principal=kudu/127.2.120.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:45801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:36.916052 24684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:36.917671 24684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:36.933524 24689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:36.933611 24693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:36.933564 24691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:36.935369 24684 server_base.cc:1034] running on GCE node
I20260811 00:31:36.936084 24684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:36.937222 24684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:36.938490 24684 hybrid_clock.cc:648] HybridClock initialized: now 1786408296938416 us; error 44 us; skew 500 ppm
I20260811 00:31:36.938851 24684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:36 dist-test-slave-fd7l krb5kdc[24560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408296, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:36.942555 24684 init.cc:377] Logged in from keytab as kudu/127.2.120.1@KRBTEST.COM (short username kudu)
I20260811 00:31:36.944192 24684 webserver.cc:460] Webserver started at http://127.2.120.1:43413/ using document root <none> and password file <none>
I20260811 00:31:36.944983 24684 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:36.945122 24684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:36.945477 24684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:36.948977 24684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "20b969d93d81430a90b8f078fbaa4f9b"
format_stamp: "Formatted at 2026-08-11 00:31:36 on dist-test-slave-fd7l"
server_key: "4f66b6cb785f5ef76802ae598b41ebb0"
server_key_iv: "0a31d587bdca7d45167a0fd140ebd451"
server_key_version: "encryptionkey@0"
I20260811 00:31:36.949833 24684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "20b969d93d81430a90b8f078fbaa4f9b"
format_stamp: "Formatted at 2026-08-11 00:31:36 on dist-test-slave-fd7l"
server_key: "4f66b6cb785f5ef76802ae598b41ebb0"
server_key_iv: "0a31d587bdca7d45167a0fd140ebd451"
server_key_version: "encryptionkey@0"
I20260811 00:31:36.956070 24684 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260811 00:31:36.960493 24702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:36.962217 24684 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:31:36.962441 24684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "20b969d93d81430a90b8f078fbaa4f9b"
format_stamp: "Formatted at 2026-08-11 00:31:36 on dist-test-slave-fd7l"
server_key: "4f66b6cb785f5ef76802ae598b41ebb0"
server_key_iv: "0a31d587bdca7d45167a0fd140ebd451"
server_key_version: "encryptionkey@0"
I20260811 00:31:36.962714 24684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:36.979353 24684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:36.984274 24684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:36.984678 24684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:36.985750 24684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:36.987283 24684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:36.987398 24684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:36.987568 24684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:36.987655 24684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:37.029986 24684 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:35151
I20260811 00:31:37.030051 24832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:35151 every 8 connection(s)
I20260811 00:31:37.031644 24684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:31:37.041491 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 24684
I20260811 00:31:37.041790 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:31:37.042699 2528 external_mini_cluster.cc:1428] Setting key 654c9ce1527574dd42288473a16bc19a
Aug 11 00:31:37 dist-test-slave-fd7l krb5kdc[24560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408296, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.1@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:31:37.059620 24835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:37.034922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:46787 (local address 127.2.120.62:45801)
0811 00:31:37.035475 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:37.035483 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:37.036834 (+ 1351us) server_negotiation.cc:407] Connection header received
0811 00:31:37.038841 (+ 2007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:37.038847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:37.038929 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:37.039057 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:37.041630 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:37.043109 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:37.046104 (+ 2995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:37.046623 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:37.046800 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:37.051597 (+ 4797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:37.051606 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:37.051611 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:37.051676 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:37.055327 (+ 3651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:37.056607 (+ 1280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:37.056613 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:37.056615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:37.056775 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:37.057429 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:37.057434 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:37.057436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:37.057821 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:37.058035 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:37.058894 (+ 859us) server_negotiation.cc:299] Negotiation successful
0811 00:31:37.059186 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":270,"threads_started":1}
I20260811 00:31:37.060844 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:37.035646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:45801 (local address 127.2.120.1:46787)
0811 00:31:37.036632 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:37.036685 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:37.038501 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:37.039396 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:37.039407 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:37.040389 (+ 982us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:37.041284 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:37.041310 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:37.043381 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:37.043386 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:37.045894 (+ 2508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:37.045915 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:37.046930 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:37.046941 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:37.047262 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:37.048213 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:31:37.048242 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:37.051101 (+ 2859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:37.055558 (+ 4457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:37.055573 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:37.055599 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:37.056393 (+ 794us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:37.056913 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:37.056918 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:37.056924 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:37.057152 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:37.057985 (+ 833us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:37.058007 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:37.058442 (+ 435us) client_negotiation.cc:769] Sending connection context
0811 00:31:37.058766 (+ 324us) client_negotiation.cc:240] Negotiation successful
0811 00:31:37.059423 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":382,"threads_started":1}
I20260811 00:31:37.062783 24833 heartbeater.cc:347] Connected to a master server at 127.2.120.62:45801
I20260811 00:31:37.063195 24833 heartbeater.cc:464] Registering TS with master...
I20260811 00:31:37.064196 24833 heartbeater.cc:511] Master 127.2.120.62:45801 requested a full tablet report, sending...
I20260811 00:31:37.067106 24612 ts_manager.cc:194] Registered new tserver with Master: 20b969d93d81430a90b8f078fbaa4f9b (127.2.120.1:35151)
I20260811 00:31:37.069446 24612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.1@KRBTEST.COM'} at 127.2.120.1:46787
WARNING: no policy specified for kudu/127.2.120.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
Entry for principal kudu/127.2.120.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab.
I20260811 00:31:37.123081 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:45801
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:46025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:31:37.538357 24839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:37.538697 24839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:37.538800 24839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:37.547328 24839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:37.547492 24839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:37.547614 24839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:31:37.558349 24839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:46025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.2.keytab
--principal=kudu/127.2.120.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:45801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:37.559731 24839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:37.561136 24839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:37.575806 24846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:37.575834 24844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:37.576140 24849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:37.576769 24839 server_base.cc:1034] running on GCE node
I20260811 00:31:37.577451 24839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:37.578469 24839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:37.579702 24839 hybrid_clock.cc:648] HybridClock initialized: now 1786408297579650 us; error 45 us; skew 500 ppm
I20260811 00:31:37.580075 24839 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:37 dist-test-slave-fd7l krb5kdc[24560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408297, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:37.591415 24839 init.cc:377] Logged in from keytab as kudu/127.2.120.2@KRBTEST.COM (short username kudu)
I20260811 00:31:37.595729 24839 webserver.cc:460] Webserver started at http://127.2.120.2:35521/ using document root <none> and password file <none>
I20260811 00:31:37.597829 24839 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:37.598233 24839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:37.598873 24839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:37.604786 24839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "aa5823d33eb14454be99b420027f673b"
format_stamp: "Formatted at 2026-08-11 00:31:37 on dist-test-slave-fd7l"
server_key: "ba4a5b88c288248e5f9a81cca4711f10"
server_key_iv: "d472f4e558d856153d5f4edf50ae5f97"
server_key_version: "encryptionkey@0"
I20260811 00:31:37.606730 24839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "aa5823d33eb14454be99b420027f673b"
format_stamp: "Formatted at 2026-08-11 00:31:37 on dist-test-slave-fd7l"
server_key: "ba4a5b88c288248e5f9a81cca4711f10"
server_key_iv: "d472f4e558d856153d5f4edf50ae5f97"
server_key_version: "encryptionkey@0"
I20260811 00:31:37.615595 24839 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260811 00:31:37.619985 24856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:37.621569 24839 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:31:37.621873 24839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "aa5823d33eb14454be99b420027f673b"
format_stamp: "Formatted at 2026-08-11 00:31:37 on dist-test-slave-fd7l"
server_key: "ba4a5b88c288248e5f9a81cca4711f10"
server_key_iv: "d472f4e558d856153d5f4edf50ae5f97"
server_key_version: "encryptionkey@0"
I20260811 00:31:37.622123 24839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:37.638758 24839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:37.643668 24839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:37.644004 24839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:37.645316 24839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:37.646844 24839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:37.646957 24839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:37.647135 24839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:37.647202 24839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:37.688000 24839 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:44297
I20260811 00:31:37.688105 24985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:44297 every 8 connection(s)
I20260811 00:31:37.689456 24839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:31:37.692713 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 24839
I20260811 00:31:37.692938 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:31:37.693583 2528 external_mini_cluster.cc:1428] Setting key 906071a2e8a20ea475b0abe68e5b353a
Aug 11 00:31:37 dist-test-slave-fd7l krb5kdc[24560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408297, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.2@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:31:37.715427 24988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:37.692728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:41523 (local address 127.2.120.62:45801)
0811 00:31:37.693333 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:37.693341 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:37.694590 (+ 1249us) server_negotiation.cc:407] Connection header received
0811 00:31:37.696870 (+ 2280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:37.696878 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:37.696996 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:37.697129 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:37.698885 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:37.700186 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:37.702341 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:37.703260 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:37.703425 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:37.707724 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:37.707732 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:37.707735 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:37.707780 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:37.711414 (+ 3634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:37.712434 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:37.712440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:37.712442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:37.712667 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:37.713275 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:37.713281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:37.713284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:37.713632 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:37.713825 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:37.714701 (+ 876us) server_negotiation.cc:299] Negotiation successful
0811 00:31:37.714986 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":288,"threads_started":1}
I20260811 00:31:37.716722 24990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:37.693339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:45801 (local address 127.2.120.2:41523)
0811 00:31:37.694314 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:37.694372 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:37.696461 (+ 2089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:37.697386 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:37.697395 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:37.697997 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:37.698665 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:37.698687 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:37.700470 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:37.700475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:37.702119 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:37.702154 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:37.703564 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:37.703571 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:37.703842 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:37.704729 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:31:37.704757 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:37.707336 (+ 2579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:37.711653 (+ 4317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:37.711664 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:37.711695 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:37.712247 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:37.712843 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:37.712850 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:37.712854 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:37.713116 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:37.713796 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:37.713806 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:37.714254 (+ 448us) client_negotiation.cc:769] Sending connection context
0811 00:31:37.714594 (+ 340us) client_negotiation.cc:240] Negotiation successful
0811 00:31:37.715248 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":389,"threads_started":1}
I20260811 00:31:37.718731 24986 heartbeater.cc:347] Connected to a master server at 127.2.120.62:45801
I20260811 00:31:37.719153 24986 heartbeater.cc:464] Registering TS with master...
I20260811 00:31:37.720247 24986 heartbeater.cc:511] Master 127.2.120.62:45801 requested a full tablet report, sending...
I20260811 00:31:37.722447 24612 ts_manager.cc:194] Registered new tserver with Master: aa5823d33eb14454be99b420027f673b (127.2.120.2:44297)
I20260811 00:31:37.723912 24612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.2@KRBTEST.COM'} at 127.2.120.2:41523
WARNING: no policy specified for kudu/127.2.120.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
Entry for principal kudu/127.2.120.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab.
I20260811 00:31:37.773281 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:45801
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:46025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:31:38.074049 24833 heartbeater.cc:503] Master 127.2.120.62:45801 was elected leader, sending a full tablet report...
W20260811 00:31:38.161926 24993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:38.162273 24993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:38.162374 24993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:38.173743 24993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:38.174024 24993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:38.174510 24993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:31:38.186187 24993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:46025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.3.keytab
--principal=kudu/127.2.120.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:45801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:38.187736 24993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:38.189302 24993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:38.204373 25000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:38.205506 25002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:38.205878 25005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:38.206471 24993 server_base.cc:1034] running on GCE node
I20260811 00:31:38.207291 24993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:38.208616 24993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:38.209893 24993 hybrid_clock.cc:648] HybridClock initialized: now 1786408298209820 us; error 80 us; skew 500 ppm
I20260811 00:31:38.210299 24993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:38 dist-test-slave-fd7l krb5kdc[24560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408298, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:38.214685 24993 init.cc:377] Logged in from keytab as kudu/127.2.120.3@KRBTEST.COM (short username kudu)
I20260811 00:31:38.217428 24993 webserver.cc:460] Webserver started at http://127.2.120.3:32831/ using document root <none> and password file <none>
I20260811 00:31:38.218521 24993 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:38.218703 24993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:38.219111 24993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:38.222694 24993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "65f638c73717486da9938c15de832ed9"
format_stamp: "Formatted at 2026-08-11 00:31:38 on dist-test-slave-fd7l"
server_key: "b7975d9fad3f1a25b04b5b49e729e605"
server_key_iv: "ac86a5bf08dc9766a69c505f2003a4c8"
server_key_version: "encryptionkey@0"
I20260811 00:31:38.223654 24993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "65f638c73717486da9938c15de832ed9"
format_stamp: "Formatted at 2026-08-11 00:31:38 on dist-test-slave-fd7l"
server_key: "b7975d9fad3f1a25b04b5b49e729e605"
server_key_iv: "ac86a5bf08dc9766a69c505f2003a4c8"
server_key_version: "encryptionkey@0"
I20260811 00:31:38.230520 24993 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 00:31:38.235216 25013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:38.237095 24993 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260811 00:31:38.237341 24993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "65f638c73717486da9938c15de832ed9"
format_stamp: "Formatted at 2026-08-11 00:31:38 on dist-test-slave-fd7l"
server_key: "b7975d9fad3f1a25b04b5b49e729e605"
server_key_iv: "ac86a5bf08dc9766a69c505f2003a4c8"
server_key_version: "encryptionkey@0"
I20260811 00:31:38.237640 24993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:38.261560 24993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:38.266844 24993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:38.267750 24993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:38.269434 24993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:38.271200 24993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:38.271293 24993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:38.271462 24993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:38.271528 24993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:38.323350 24993 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:38997
I20260811 00:31:38.323362 25142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:38997 every 8 connection(s)
I20260811 00:31:38.325834 24993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:31:38.329552 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 24993
I20260811 00:31:38.329785 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:31:38.330269 2528 external_mini_cluster.cc:1428] Setting key 9dbd77b58715300f9a617163cd03cc2f
Aug 11 00:31:38 dist-test-slave-fd7l krb5kdc[24560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408298, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.3@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:31:38.358469 25146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.331146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:48641 (local address 127.2.120.62:45801)
0811 00:31:38.335410 (+ 4264us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:38.335417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:38.336004 (+ 587us) server_negotiation.cc:407] Connection header received
0811 00:31:38.338287 (+ 2283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:38.338293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:38.338398 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:38.338547 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:38.340478 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.341502 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.343602 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.344045 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.344207 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:38.349332 (+ 5125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:38.349343 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:38.349351 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:38.349415 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:38.354080 (+ 4665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:38.355160 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:38.355170 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:38.355175 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:38.355378 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:38.355926 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:38.355934 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:38.355940 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:38.356414 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:38.356666 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:38.357635 (+ 969us) server_negotiation.cc:299] Negotiation successful
0811 00:31:38.357939 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4083,"thread_start_us":3933,"threads_started":1}
I20260811 00:31:38.360038 25147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.330915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:45801 (local address 127.2.120.3:48641)
0811 00:31:38.335812 (+ 4897us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:38.335862 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:38.337930 (+ 2068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:38.338858 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:38.338867 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:38.339492 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:38.340264 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:38.340288 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.341702 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.341706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:38.343375 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:38.343392 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.344395 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.344404 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:38.344759 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:38.345716 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:31:38.345743 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:38.349047 (+ 3304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:38.354355 (+ 5308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:38.354365 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:38.354382 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:38.354983 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:38.355560 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:31:38.355564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:31:38.355566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:31:38.355791 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:31:38.356691 (+ 900us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:38.356699 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:38.357138 (+ 439us) client_negotiation.cc:769] Sending connection context
0811 00:31:38.357518 (+ 380us) client_negotiation.cc:240] Negotiation successful
0811 00:31:38.358218 (+ 700us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4580,"thread_start_us":4262,"threads_started":1}
I20260811 00:31:38.362051 25144 heartbeater.cc:347] Connected to a master server at 127.2.120.62:45801
I20260811 00:31:38.362970 25144 heartbeater.cc:464] Registering TS with master...
I20260811 00:31:38.363947 25144 heartbeater.cc:511] Master 127.2.120.62:45801 requested a full tablet report, sending...
I20260811 00:31:38.366019 24612 ts_manager.cc:194] Registered new tserver with Master: 65f638c73717486da9938c15de832ed9 (127.2.120.3:38997)
I20260811 00:31:38.367168 24612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.120.3@KRBTEST.COM'} at 127.2.120.3:48641
I20260811 00:31:38.375982 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:31:38.407277 25146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.383737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58082 (local address 127.2.120.62:45801)
0811 00:31:38.384157 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:38.384169 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:38.385259 (+ 1090us) server_negotiation.cc:407] Connection header received
0811 00:31:38.385726 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:38.385732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:38.385829 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:38.385958 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:38.388863 (+ 2905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.389812 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.392410 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.392982 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.393142 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:38.398885 (+ 5743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:38.398893 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:38.398897 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:38.398948 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:38.402978 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:38.404401 (+ 1423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:38.404407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:38.404410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:38.404694 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:38.405303 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:38.405307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:38.405309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:38.405715 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:38.406009 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:38.406605 (+ 596us) server_negotiation.cc:299] Negotiation successful
0811 00:31:38.406866 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260811 00:31:38.413213 24612 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:31:38.422435 24612 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:31:38.469182 25163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.437739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:38997 (local address 127.0.0.1:37522)
0811 00:31:38.450351 (+ 12612us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:38.450419 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:38.450715 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:38.457416 (+ 6701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:38.457430 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:38.457492 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:38.458092 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:38.458113 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.461498 (+ 3385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.461509 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:38.464834 (+ 3325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:38.464858 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.467615 (+ 2757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.467624 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:38.468036 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:38.468134 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 00:31:38.468282 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 00:31:38.468586 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12242,"mutex_wait_us":1,"threads_started":1}
I20260811 00:31:38.471359 25161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.438367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37522 (local address 127.2.120.3:38997)
0811 00:31:38.456135 (+ 17768us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:38.456149 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:38.456201 (+ 52us) server_negotiation.cc:407] Connection header received
0811 00:31:38.456361 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:38.456374 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:38.456763 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:38.458527 (+ 1764us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:38.458919 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.461248 (+ 2329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.465135 (+ 3887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.467299 (+ 2164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.470323 (+ 3024us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:38.470636 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:38.470729 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 00:31:38.470942 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17451,"thread_start_us":18593,"threads_started":1}
I20260811 00:31:38.474938 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.439026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50262 (local address 127.2.120.1:35151)
0811 00:31:38.448547 (+ 9521us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:38.448562 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:38.461159 (+ 12597us) server_negotiation.cc:407] Connection header received
0811 00:31:38.461895 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:38.461911 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:38.462639 (+ 728us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:38.462938 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:38.464466 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.466818 (+ 2352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.471486 (+ 4668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.473218 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.473329 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:38.473582 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:38.474379 (+ 797us) server_negotiation.cc:299] Negotiation successful
0811 00:31:38.474580 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8507,"thread_start_us":8301,"threads_started":1}
I20260811 00:31:38.475883 25159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.438267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:35151 (local address 127.0.0.1:50262)
0811 00:31:38.460938 (+ 22671us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:38.460971 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:38.461210 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:38.463480 (+ 2270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:38.463486 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:38.463528 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:38.464169 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:38.464187 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.467190 (+ 3003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.467199 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:38.470706 (+ 3507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:38.470732 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.473802 (+ 3070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.473808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:38.474124 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:38.474171 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 00:31:38.475284 (+ 1113us) client_negotiation.cc:240] Negotiation successful
0811 00:31:38.475464 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22450,"mutex_wait_us":34,"thread_start_us":23090,"threads_started":1}
I20260811 00:31:38.477694 25068 tablet_service.cc:1467] Processing CreateTablet for tablet ee342b4cae9d47b983458fc866c584cd (DEFAULT_TABLE table=test-table [id=63fb6b4b5a2b42dcb90077ea217bca5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:31:38.480063 25068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee342b4cae9d47b983458fc866c584cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:38.483711 24756 tablet_service.cc:1467] Processing CreateTablet for tablet ee342b4cae9d47b983458fc866c584cd (DEFAULT_TABLE table=test-table [id=63fb6b4b5a2b42dcb90077ea217bca5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:31:38.488940 24756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee342b4cae9d47b983458fc866c584cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:38.489252 25163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.449370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:44297 (local address 127.0.0.1:52118)
0811 00:31:38.469881 (+ 20511us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:38.469917 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:38.470095 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:38.471173 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:38.471181 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:38.471216 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:38.471482 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:38.471501 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.474017 (+ 2516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.474025 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:38.476647 (+ 2622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:38.476675 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.488237 (+ 11562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.488244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:38.488494 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:38.488570 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 00:31:38.488688 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 00:31:38.488877 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20321,"mutex_wait_us":27,"thread_start_us":944,"threads_started":1}
I20260811 00:31:38.489985 25164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.450136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52118 (local address 127.2.120.2:44297)
0811 00:31:38.457042 (+ 6906us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:38.457050 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:38.470051 (+ 13001us) server_negotiation.cc:407] Connection header received
0811 00:31:38.470294 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:38.470304 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:38.470651 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:38.470895 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:38.471681 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.473685 (+ 2004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.476925 (+ 3240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.487311 (+ 10386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.487455 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:38.487687 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:38.488780 (+ 1093us) server_negotiation.cc:299] Negotiation successful
0811 00:31:38.489660 (+ 880us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6682,"thread_start_us":6570,"threads_started":1}
I20260811 00:31:38.494484 24912 tablet_service.cc:1467] Processing CreateTablet for tablet ee342b4cae9d47b983458fc866c584cd (DEFAULT_TABLE table=test-table [id=63fb6b4b5a2b42dcb90077ea217bca5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:31:38.496644 24912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee342b4cae9d47b983458fc866c584cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:38.497431 25165 tablet_bootstrap.cc:492] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9: Bootstrap starting.
I20260811 00:31:38.502878 25165 tablet_bootstrap.cc:654] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:38.505257 25165 log.cc:824] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:38.505591 25167 tablet_bootstrap.cc:492] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b: Bootstrap starting.
I20260811 00:31:38.510871 25167 tablet_bootstrap.cc:654] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:38.511897 25165 tablet_bootstrap.cc:492] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9: No bootstrap required, opened a new log
I20260811 00:31:38.511960 25169 tablet_bootstrap.cc:492] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b: Bootstrap starting.
I20260811 00:31:38.512264 25165 ts_tablet_manager.cc:1397] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260811 00:31:38.512465 25167 log.cc:824] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:38.515503 25167 tablet_bootstrap.cc:492] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b: No bootstrap required, opened a new log
I20260811 00:31:38.515846 25167 ts_tablet_manager.cc:1397] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260811 00:31:38.516311 25169 tablet_bootstrap.cc:654] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:38.517877 25169 log.cc:824] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:38.522387 25169 tablet_bootstrap.cc:492] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b: No bootstrap required, opened a new log
I20260811 00:31:38.522761 25169 ts_tablet_manager.cc:1397] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 00:31:38.523859 25165 raft_consensus.cc:348] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f638c73717486da9938c15de832ed9" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38997 } } peers { permanent_uuid: "20b969d93d81430a90b8f078fbaa4f9b" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 35151 } } peers { permanent_uuid: "aa5823d33eb14454be99b420027f673b" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 44297 } }
I20260811 00:31:38.524376 25165 raft_consensus.cc:374] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:38.524500 25165 raft_consensus.cc:729] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65f638c73717486da9938c15de832ed9, State: Initialized, Role: FOLLOWER
I20260811 00:31:38.524873 25167 raft_consensus.cc:348] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f638c73717486da9938c15de832ed9" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38997 } } peers { permanent_uuid: "20b969d93d81430a90b8f078fbaa4f9b" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 35151 } } peers { permanent_uuid: "aa5823d33eb14454be99b420027f673b" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 44297 } }
I20260811 00:31:38.525326 25167 raft_consensus.cc:374] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:38.525452 25167 raft_consensus.cc:729] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20b969d93d81430a90b8f078fbaa4f9b, State: Initialized, Role: FOLLOWER
I20260811 00:31:38.525259 25165 consensus_queue.cc:260] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f638c73717486da9938c15de832ed9" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38997 } } peers { permanent_uuid: "20b969d93d81430a90b8f078fbaa4f9b" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 35151 } } peers { permanent_uuid: "aa5823d33eb14454be99b420027f673b" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 44297 } }
I20260811 00:31:38.526233 25167 consensus_queue.cc:260] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f638c73717486da9938c15de832ed9" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38997 } } peers { permanent_uuid: "20b969d93d81430a90b8f078fbaa4f9b" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 35151 } } peers { permanent_uuid: "aa5823d33eb14454be99b420027f673b" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 44297 } }
I20260811 00:31:38.526671 25144 heartbeater.cc:503] Master 127.2.120.62:45801 was elected leader, sending a full tablet report...
I20260811 00:31:38.527549 25165 ts_tablet_manager.cc:1428] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9: Time spent starting tablet: real 0.015s user 0.017s sys 0.000s
I20260811 00:31:38.529718 25167 ts_tablet_manager.cc:1428] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260811 00:31:38.534279 25169 raft_consensus.cc:348] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f638c73717486da9938c15de832ed9" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38997 } } peers { permanent_uuid: "20b969d93d81430a90b8f078fbaa4f9b" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 35151 } } peers { permanent_uuid: "aa5823d33eb14454be99b420027f673b" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 44297 } }
I20260811 00:31:38.534924 25169 raft_consensus.cc:374] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:38.535048 25169 raft_consensus.cc:729] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa5823d33eb14454be99b420027f673b, State: Initialized, Role: FOLLOWER
W20260811 00:31:38.535053 24834 tablet.cc:2369] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:31:38.535822 25169 consensus_queue.cc:260] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f638c73717486da9938c15de832ed9" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38997 } } peers { permanent_uuid: "20b969d93d81430a90b8f078fbaa4f9b" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 35151 } } peers { permanent_uuid: "aa5823d33eb14454be99b420027f673b" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 44297 } }
I20260811 00:31:38.537283 24986 heartbeater.cc:503] Master 127.2.120.62:45801 was elected leader, sending a full tablet report...
I20260811 00:31:38.538111 25169 ts_tablet_manager.cc:1428] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260811 00:31:38.578745 25145 tablet.cc:2369] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:31:38.582505 25173 raft_consensus.cc:482] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:31:38.583055 25173 raft_consensus.cc:504] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f638c73717486da9938c15de832ed9" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38997 } } peers { permanent_uuid: "20b969d93d81430a90b8f078fbaa4f9b" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 35151 } } peers { permanent_uuid: "aa5823d33eb14454be99b420027f673b" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 44297 } }
I20260811 00:31:38.585940 25173 leader_election.cc:288] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65f638c73717486da9938c15de832ed9 (127.2.120.3:38997), aa5823d33eb14454be99b420027f673b (127.2.120.2:44297)
I20260811 00:31:38.597247 25164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.586956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:54513 (local address 127.2.120.2:44297)
0811 00:31:38.587288 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:38.587298 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:38.588345 (+ 1047us) server_negotiation.cc:407] Connection header received
0811 00:31:38.588629 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:38.588636 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:38.588734 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:38.588852 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:38.589639 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.590477 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.593940 (+ 3463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.595086 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.595428 (+ 342us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:38.595622 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:38.596451 (+ 829us) server_negotiation.cc:299] Negotiation successful
0811 00:31:38.596791 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260811 00:31:38.597224 25161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.586946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:58095 (local address 127.2.120.3:38997)
0811 00:31:38.587257 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:38.587266 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:38.588170 (+ 904us) server_negotiation.cc:407] Connection header received
0811 00:31:38.588457 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:38.588463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:38.588636 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:38.588762 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:38.589473 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.590473 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.593929 (+ 3456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.595081 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.595430 (+ 349us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:38.595616 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:38.596460 (+ 844us) server_negotiation.cc:299] Negotiation successful
0811 00:31:38.596784 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:31:38.597560 25177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.586362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:38997 (local address 127.2.120.1:58095)
0811 00:31:38.587992 (+ 1630us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:38.588016 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:38.588241 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:38.588777 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:38.588782 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:38.588812 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:38.589305 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:38.589317 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.590639 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.590645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:38.593490 (+ 2845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:38.593540 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.595655 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.595669 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:38.595992 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:38.596082 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 00:31:38.596362 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 00:31:38.596812 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1439,"mutex_wait_us":321,"thread_start_us":1436,"threads_started":1}
I20260811 00:31:38.597617 25176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.586640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:44297 (local address 127.2.120.1:54513)
0811 00:31:38.588201 (+ 1561us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:38.588223 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:38.588407 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:38.588875 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:38.588880 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:38.588910 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:38.589448 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:38.589458 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.590657 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.590663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:38.593490 (+ 2827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:38.593540 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.595714 (+ 2174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.595722 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:38.595992 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:38.596081 (+ 89us) client_negotiation.cc:769] Sending connection context
0811 00:31:38.596362 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 00:31:38.596812 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1399,"mutex_wait_us":97,"thread_start_us":699,"threads_started":1}
I20260811 00:31:38.599529 25089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee342b4cae9d47b983458fc866c584cd" candidate_uuid: "20b969d93d81430a90b8f078fbaa4f9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65f638c73717486da9938c15de832ed9" is_pre_election: true
I20260811 00:31:38.599527 24934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee342b4cae9d47b983458fc866c584cd" candidate_uuid: "20b969d93d81430a90b8f078fbaa4f9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa5823d33eb14454be99b420027f673b" is_pre_election: true
I20260811 00:31:38.600332 25089 raft_consensus.cc:2454] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20b969d93d81430a90b8f078fbaa4f9b in term 0.
I20260811 00:31:38.600335 24934 raft_consensus.cc:2454] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20b969d93d81430a90b8f078fbaa4f9b in term 0.
I20260811 00:31:38.601661 24706 leader_election.cc:302] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20b969d93d81430a90b8f078fbaa4f9b, 65f638c73717486da9938c15de832ed9; no voters:
I20260811 00:31:38.602396 25173 raft_consensus.cc:2781] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:31:38.602639 25173 raft_consensus.cc:482] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:31:38.602782 25173 raft_consensus.cc:3037] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:38.606081 25173 raft_consensus.cc:504] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f638c73717486da9938c15de832ed9" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38997 } } peers { permanent_uuid: "20b969d93d81430a90b8f078fbaa4f9b" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 35151 } } peers { permanent_uuid: "aa5823d33eb14454be99b420027f673b" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 44297 } }
I20260811 00:31:38.607441 25173 leader_election.cc:288] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [CANDIDATE]: Term 1 election: Requested vote from peers 65f638c73717486da9938c15de832ed9 (127.2.120.3:38997), aa5823d33eb14454be99b420027f673b (127.2.120.2:44297)
I20260811 00:31:38.608333 25089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee342b4cae9d47b983458fc866c584cd" candidate_uuid: "20b969d93d81430a90b8f078fbaa4f9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65f638c73717486da9938c15de832ed9"
I20260811 00:31:38.608484 24934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee342b4cae9d47b983458fc866c584cd" candidate_uuid: "20b969d93d81430a90b8f078fbaa4f9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa5823d33eb14454be99b420027f673b"
I20260811 00:31:38.608814 25089 raft_consensus.cc:3037] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:38.608860 24934 raft_consensus.cc:3037] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:38.613632 25089 raft_consensus.cc:2454] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20b969d93d81430a90b8f078fbaa4f9b in term 1.
I20260811 00:31:38.613628 24934 raft_consensus.cc:2454] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20b969d93d81430a90b8f078fbaa4f9b in term 1.
I20260811 00:31:38.614691 24705 leader_election.cc:302] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20b969d93d81430a90b8f078fbaa4f9b, aa5823d33eb14454be99b420027f673b; no voters:
I20260811 00:31:38.615386 25173 raft_consensus.cc:2781] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:38.616851 25173 raft_consensus.cc:686] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [term 1 LEADER]: Becoming Leader. State: Replica: 20b969d93d81430a90b8f078fbaa4f9b, State: Running, Role: LEADER
I20260811 00:31:38.617849 25173 consensus_queue.cc:237] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f638c73717486da9938c15de832ed9" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38997 } } peers { permanent_uuid: "20b969d93d81430a90b8f078fbaa4f9b" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 35151 } } peers { permanent_uuid: "aa5823d33eb14454be99b420027f673b" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 44297 } }
I20260811 00:31:38.626061 24611 catalog_manager.cc:5495] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 20b969d93d81430a90b8f078fbaa4f9b (127.2.120.1). New cstate: current_term: 1 leader_uuid: "20b969d93d81430a90b8f078fbaa4f9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f638c73717486da9938c15de832ed9" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20b969d93d81430a90b8f078fbaa4f9b" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 35151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa5823d33eb14454be99b420027f673b" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 44297 } health_report { overall_health: UNKNOWN } } }
I20260811 00:31:38.657354 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.650267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50266 (local address 127.2.120.1:35151)
0811 00:31:38.650620 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:38.650626 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:38.650706 (+ 80us) server_negotiation.cc:407] Connection header received
0811 00:31:38.650987 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:38.650992 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:38.651119 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:38.651318 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:31:38.651865 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.652834 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.654449 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.654915 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.655088 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:38.655720 (+ 632us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:31:38.656330 (+ 610us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:31:38.656513 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:38.656669 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 00:31:38.656770 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
I20260811 00:31:38.668905 24934 raft_consensus.cc:1261] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b [term 1 FOLLOWER]: Refusing update from remote peer 20b969d93d81430a90b8f078fbaa4f9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:31:38.668905 25089 raft_consensus.cc:1261] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9 [term 1 FOLLOWER]: Refusing update from remote peer 20b969d93d81430a90b8f078fbaa4f9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:31:38.670567 25178 consensus_queue.cc:1035] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa5823d33eb14454be99b420027f673b" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 44297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:31:38.671098 25173 consensus_queue.cc:1035] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "65f638c73717486da9938c15de832ed9" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 38997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:31:38.712041 24610 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:31:38.712468 24610 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:31:38.717885 24610 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b82e7457a935450db3369c7a35cae3dd: Sending DeleteTablet for 3 replicas of tablet ee342b4cae9d47b983458fc866c584cd
I20260811 00:31:38.719434 25068 tablet_service.cc:1514] Processing DeleteTablet for tablet ee342b4cae9d47b983458fc866c584cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:31:38 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:37522
I20260811 00:31:38.719517 24756 tablet_service.cc:1514] Processing DeleteTablet for tablet ee342b4cae9d47b983458fc866c584cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:31:38 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:50262
I20260811 00:31:38.719645 24912 tablet_service.cc:1514] Processing DeleteTablet for tablet ee342b4cae9d47b983458fc866c584cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:31:38 UTC) from {username='kudu', principal='kudu/127.2.120.62@KRBTEST.COM'} at 127.0.0.1:52118
I20260811 00:31:38.721889 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 24684
I20260811 00:31:38.739419 25192 tablet_replica.cc:330] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9: stopping tablet replica
I20260811 00:31:38.739861 25193 tablet_replica.cc:330] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b: stopping tablet replica
I20260811 00:31:38.740051 25192 raft_consensus.cc:2229] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:31:38.740365 25193 raft_consensus.cc:2229] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:31:38.741158 25193 raft_consensus.cc:2258] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:31:38.742092 25192 raft_consensus.cc:2258] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:31:38.744009 25193 ts_tablet_manager.cc:1905] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:31:38.746024 25192 ts_tablet_manager.cc:1905] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 00:31:38 dist-test-slave-fd7l krb5kdc[24560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.1@KRBTEST.COM
I20260811 00:31:38.746842 25194 tablet_replica.cc:330] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b: stopping tablet replica
I20260811 00:31:38.747421 25194 raft_consensus.cc:2229] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:31:38.748377 25194 raft_consensus.cc:2258] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:31:38.752571 25193 ts_tablet_manager.cc:1918] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:31:38.752799 25193 log.cc:1196] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/ee342b4cae9d47b983458fc866c584cd
I20260811 00:31:38.753401 25193 ts_tablet_manager.cc:1939] T ee342b4cae9d47b983458fc866c584cd P aa5823d33eb14454be99b420027f673b: Deleting consensus metadata
I20260811 00:31:38.755155 25192 ts_tablet_manager.cc:1918] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:31:38.755457 25192 log.cc:1196] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/ee342b4cae9d47b983458fc866c584cd
I20260811 00:31:38.755679 24597 catalog_manager.cc:4841] TS aa5823d33eb14454be99b420027f673b (127.2.120.2:44297): tablet ee342b4cae9d47b983458fc866c584cd (table test-table [id=63fb6b4b5a2b42dcb90077ea217bca5a]) successfully deleted
I20260811 00:31:38.756261 25192 ts_tablet_manager.cc:1939] T ee342b4cae9d47b983458fc866c584cd P 65f638c73717486da9938c15de832ed9: Deleting consensus metadata
I20260811 00:31:38.758241 25194 ts_tablet_manager.cc:1905] T ee342b4cae9d47b983458fc866c584cd P 20b969d93d81430a90b8f078fbaa4f9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:31:38.758540 24599 catalog_manager.cc:4841] TS 65f638c73717486da9938c15de832ed9 (127.2.120.3:38997): tablet ee342b4cae9d47b983458fc866c584cd (table test-table [id=63fb6b4b5a2b42dcb90077ea217bca5a]) successfully deleted
I20260811 00:31:38.758876 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.725062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50272 (local address 127.2.120.1:35151)
0811 00:31:38.725369 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:38.725380 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:38.728405 (+ 3025us) server_negotiation.cc:407] Connection header received
0811 00:31:38.729789 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:38.729797 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:38.729918 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:38.730022 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:38.734429 (+ 4407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.735731 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.739853 (+ 4122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.740752 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.740875 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:38.749594 (+ 8719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:38.749614 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:38.749636 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:38.749701 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:38.754312 (+ 4611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:38.755557 (+ 1245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:38.755571 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:38.755574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:38.755833 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:38.756425 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:38.756444 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:38.756448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:38.757054 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:38.757378 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:38.758108 (+ 730us) server_negotiation.cc:299] Negotiation successful
0811 00:31:38.758445 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260811 00:31:38.761036 24828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:38.866672 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 24684
I20260811 00:31:38.891723 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 24839
W20260811 00:31:38.891637 24595 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.120.1:35151 (error 108)
I20260811 00:31:38.894375 25163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.893530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:35151 (local address 127.0.0.1:50276)
0811 00:31:38.893788 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:38.894005 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:35151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
W20260811 00:31:38.894683 24595 catalog_manager.cc:4584] TS 20b969d93d81430a90b8f078fbaa4f9b (127.2.120.1:35151): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ee342b4cae9d47b983458fc866c584cd: Network error: Client connection negotiation failed: client connection to 127.2.120.1:35151: connect: Connection refused (error 111)
Aug 11 00:31:38 dist-test-slave-fd7l krb5kdc[24560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.2@KRBTEST.COM
I20260811 00:31:38.909559 25164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.892785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52132 (local address 127.2.120.2:44297)
0811 00:31:38.892977 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:38.892981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:38.893120 (+ 139us) server_negotiation.cc:407] Connection header received
0811 00:31:38.893404 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:38.893413 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:38.893518 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:38.893655 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:38.895485 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.896250 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.897684 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:38.898160 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:38.898282 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:38.902991 (+ 4709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:38.903019 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:38.903046 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:38.903095 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:38.906267 (+ 3172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:38.907233 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:38.907241 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:38.907244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:38.907413 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:38.907875 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:38.907878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:38.907880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:38.908286 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:38.908640 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:38.909079 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 00:31:38.909273 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:31:38.910341 24981 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:38.948638 25163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.947918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:35151 (local address 127.0.0.1:50278)
0811 00:31:38.948256 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:38.948381 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:35151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 00:31:38.950050 25163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:38.949544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:35151 (local address 127.0.0.1:50282)
0811 00:31:38.949740 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:38.949837 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:35151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 00:31:39.006794 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 24839
I20260811 00:31:39.010845 25163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:39.010181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:35151 (local address 127.0.0.1:50298)
0811 00:31:39.010488 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:39.010620 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:35151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 00:31:39.012283 25163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:39.011739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:35151 (local address 127.0.0.1:50310)
0811 00:31:39.011962 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:39.012084 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:35151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 00:31:39.030828 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 24993
Aug 11 00:31:39 dist-test-slave-fd7l krb5kdc[24560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.3@KRBTEST.COM
I20260811 00:31:39.052323 25161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:39.031725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37530 (local address 127.2.120.3:38997)
0811 00:31:39.031919 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:39.031924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:39.031945 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:31:39.032134 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:39.032138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:39.032222 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:39.032314 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:39.033949 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:39.034935 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:39.037139 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:39.037839 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:39.038008 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:39.044022 (+ 6014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:39.044043 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:39.044065 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:39.044121 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:39.048438 (+ 4317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:39.049683 (+ 1245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:39.049692 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:39.049694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:39.049884 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:39.050325 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:39.050330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:39.050332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:39.050778 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:39.051070 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:39.051757 (+ 687us) server_negotiation.cc:299] Negotiation successful
0811 00:31:39.051995 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 00:31:39.053092 25138 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:39.093400 25163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:39.092817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:35151 (local address 127.0.0.1:50318)
0811 00:31:39.093085 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:39.093208 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:35151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 00:31:39.094789 25163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:39.094241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:35151 (local address 127.0.0.1:50322)
0811 00:31:39.094490 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:39.094603 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:35151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260811 00:31:39.147357 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 24993
I20260811 00:31:39.170157 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 24576
I20260811 00:31:39.171393 24647 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:39.290172 25163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:39.289486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:35151 (local address 127.0.0.1:50334)
0811 00:31:39.289820 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:39.289933 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:35151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
I20260811 00:31:39.290263 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 24576
2026-08-11T00:31:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5134 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T00:31:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:31:39Z Disabled control of system clock
I20260811 00:31:39.393532 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:32801
--webserver_interface=127.2.120.62
--webserver_port=0
--builtin_ntp_servers=127.2.120.20:33253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:32801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 00:31:39.813166 25205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:39.813516 25205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:39.813613 25205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:39.821621 25205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:31:39.821767 25205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:39.821807 25205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:39.821859 25205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:31:39.821906 25205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:31:39.832891 25205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:33253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:32801
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:32801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:39.834908 25205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:39.837503 25205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:39.852674 25214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:39.852716 25210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:39.852619 25212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:39.855942 25205 server_base.cc:1034] running on GCE node
I20260811 00:31:39.857107 25205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:39.859134 25205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:39.860476 25205 hybrid_clock.cc:648] HybridClock initialized: now 1786408299860419 us; error 87 us; skew 500 ppm
I20260811 00:31:39.861030 25205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:31:39.864061 25205 webserver.cc:460] Webserver started at http://127.2.120.62:40903/ using document root <none> and password file <none>
I20260811 00:31:39.865005 25205 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:39.865175 25205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:39.865581 25205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:39.869350 25205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "44b453cdca574943ac645f45641d9d1f"
format_stamp: "Formatted at 2026-08-11 00:31:39 on dist-test-slave-fd7l"
server_key: "2e12f78f3c8d358f45078f2e7f808bec"
server_key_iv: "07b0e74450699a94491ae8fbd1882d94"
server_key_version: "encryptionkey@0"
I20260811 00:31:39.870332 25205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "44b453cdca574943ac645f45641d9d1f"
format_stamp: "Formatted at 2026-08-11 00:31:39 on dist-test-slave-fd7l"
server_key: "2e12f78f3c8d358f45078f2e7f808bec"
server_key_iv: "07b0e74450699a94491ae8fbd1882d94"
server_key_version: "encryptionkey@0"
I20260811 00:31:39.877161 25205 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 00:31:39.881886 25222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:39.883963 25205 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:31:39.884250 25205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "44b453cdca574943ac645f45641d9d1f"
format_stamp: "Formatted at 2026-08-11 00:31:39 on dist-test-slave-fd7l"
server_key: "2e12f78f3c8d358f45078f2e7f808bec"
server_key_iv: "07b0e74450699a94491ae8fbd1882d94"
server_key_version: "encryptionkey@0"
I20260811 00:31:39.884552 25205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:39.903820 25205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:39.904909 25205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:39.905254 25205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:39.926648 25205 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:32801
I20260811 00:31:39.926669 25279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:32801 every 8 connection(s)
I20260811 00:31:39.928196 25205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:31:39.930227 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 25205
I20260811 00:31:39.930426 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:31:39.931020 2528 external_mini_cluster.cc:1428] Setting key 0438dda516a71fa56f2da50455aaa1c6
I20260811 00:31:39.933792 25280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:39.950347 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:39.934073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40098 (local address 127.2.120.62:32801)
0811 00:31:39.935344 (+ 1271us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:39.935357 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:39.935408 (+ 51us) server_negotiation.cc:407] Connection header received
0811 00:31:39.937401 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:39.937439 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:39.938068 (+ 629us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:39.938650 (+ 582us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:39.939861 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:39.941576 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:39.943774 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:39.944751 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:39.944942 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:39.946710 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:39.946724 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:39.946755 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:39.947823 (+ 1068us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:39.947969 (+ 146us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:39.948005 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:39.948182 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:39.948719 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 00:31:39.949185 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":499,"threads_started":1}
I20260811 00:31:39.950650 25280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f: Bootstrap starting.
I20260811 00:31:39.955760 25280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:39.957546 25280 log.cc:824] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:39.962335 25280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f: No bootstrap required, opened a new log
I20260811 00:31:39.971725 25280 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b453cdca574943ac645f45641d9d1f" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 32801 } }
I20260811 00:31:39.972180 25280 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:39.972312 25280 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44b453cdca574943ac645f45641d9d1f, State: Initialized, Role: FOLLOWER
I20260811 00:31:39.973035 25280 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b453cdca574943ac645f45641d9d1f" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 32801 } }
I20260811 00:31:39.973305 25280 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:31:39.973479 25280 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:31:39.973702 25280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:39.976262 25280 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b453cdca574943ac645f45641d9d1f" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 32801 } }
I20260811 00:31:39.976955 25280 leader_election.cc:302] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44b453cdca574943ac645f45641d9d1f; no voters:
I20260811 00:31:39.977681 25280 leader_election.cc:288] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:31:39.977898 25286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:39.978788 25286 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f [term 1 LEADER]: Becoming Leader. State: Replica: 44b453cdca574943ac645f45641d9d1f, State: Running, Role: LEADER
I20260811 00:31:39.979485 25286 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b453cdca574943ac645f45641d9d1f" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 32801 } }
I20260811 00:31:39.980183 25280 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:31:39.984843 25288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44b453cdca574943ac645f45641d9d1f. Latest consensus state: current_term: 1 leader_uuid: "44b453cdca574943ac645f45641d9d1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b453cdca574943ac645f45641d9d1f" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 32801 } } }
I20260811 00:31:39.985664 25288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:39.984691 25287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44b453cdca574943ac645f45641d9d1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b453cdca574943ac645f45641d9d1f" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 32801 } } }
I20260811 00:31:39.986270 25287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:39.988686 25293 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:31:39.996726 25293 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:31:40.009022 25293 catalog_manager.cc:1347] Generated new cluster ID: ed647787c39f4ae0bb945c2d44fe39b2
I20260811 00:31:40.009196 25293 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:31:40.037194 25293 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:31:40.040458 25293 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:31:40.051059 25293 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f: Generated new TSK 0
I20260811 00:31:40.052353 25293 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:31:40.068127 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.1:0
--local_ip_for_outbound_sockets=127.2.120.1
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:32801
--builtin_ntp_servers=127.2.120.20:33253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 00:31:40.474327 25306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:40.474669 25306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:40.474824 25306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:40.483451 25306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:40.483606 25306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:40.483793 25306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.1
I20260811 00:31:40.496863 25306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:33253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.120.1
--webserver_port=0
--tserver_master_addrs=127.2.120.62:32801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.1
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:40.498508 25306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:40.500665 25306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:40.516296 25315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:40.516341 25316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:40.516270 25318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:40.517201 25306 server_base.cc:1034] running on GCE node
I20260811 00:31:40.517871 25306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:40.518967 25306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:40.520254 25306 hybrid_clock.cc:648] HybridClock initialized: now 1786408300520206 us; error 68 us; skew 500 ppm
I20260811 00:31:40.520718 25306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:31:40.523360 25306 webserver.cc:460] Webserver started at http://127.2.120.1:34103/ using document root <none> and password file <none>
I20260811 00:31:40.524147 25306 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:40.524295 25306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:40.524729 25306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:40.527896 25306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-0/data/instance:
uuid: "8dfd44fa95df4a3f89f36677399a18d7"
format_stamp: "Formatted at 2026-08-11 00:31:40 on dist-test-slave-fd7l"
server_key: "fa80d9636ee1fb1ad6bdddc8e0afdd0d"
server_key_iv: "b59036ff782da69bdf9d9f7db931a7e2"
server_key_version: "encryptionkey@0"
I20260811 00:31:40.528905 25306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance:
uuid: "8dfd44fa95df4a3f89f36677399a18d7"
format_stamp: "Formatted at 2026-08-11 00:31:40 on dist-test-slave-fd7l"
server_key: "fa80d9636ee1fb1ad6bdddc8e0afdd0d"
server_key_iv: "b59036ff782da69bdf9d9f7db931a7e2"
server_key_version: "encryptionkey@0"
I20260811 00:31:40.534947 25306 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260811 00:31:40.539060 25324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:40.540798 25306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:31:40.541014 25306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-0/wal
uuid: "8dfd44fa95df4a3f89f36677399a18d7"
format_stamp: "Formatted at 2026-08-11 00:31:40 on dist-test-slave-fd7l"
server_key: "fa80d9636ee1fb1ad6bdddc8e0afdd0d"
server_key_iv: "b59036ff782da69bdf9d9f7db931a7e2"
server_key_version: "encryptionkey@0"
I20260811 00:31:40.541270 25306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:40.557971 25306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:40.558907 25306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:40.559221 25306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:40.560300 25306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:40.561949 25306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:40.562078 25306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:40.562232 25306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:40.562321 25306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:40.600835 25306 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.1:33873
I20260811 00:31:40.600909 25451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.1:33873 every 8 connection(s)
I20260811 00:31:40.602356 25306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-0/data/info.pb
I20260811 00:31:40.603097 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 25306
I20260811 00:31:40.603308 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-0/wal/instance
I20260811 00:31:40.603987 2528 external_mini_cluster.cc:1428] Setting key d0aaf34944cbd130fc97f7e2ca85f727
I20260811 00:31:40.607689 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.2:0
--local_ip_for_outbound_sockets=127.2.120.2
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:32801
--builtin_ntp_servers=127.2.120.20:33253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 00:31:40.618213 25455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:40.605666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.1:51867 (local address 127.2.120.62:32801)
0811 00:31:40.606204 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:40.606212 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:40.607624 (+ 1412us) server_negotiation.cc:407] Connection header received
0811 00:31:40.609651 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:40.609657 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:40.609760 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:40.609888 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:40.611148 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:40.611930 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:40.613850 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:40.614355 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:40.614535 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:40.616960 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:40.616977 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:40.616985 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:40.617022 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:40.617086 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:40.617110 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:40.617265 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:40.617669 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 00:31:40.617810 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":303,"threads_started":1}
I20260811 00:31:40.619603 25456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:40.606378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:32801 (local address 127.2.120.1:51867)
0811 00:31:40.607458 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:40.607499 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:40.609340 (+ 1841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:40.610175 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:40.610185 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:40.610286 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:40.610943 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:40.610961 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:40.612127 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:40.612131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:40.613663 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:40.613681 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:40.614645 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:40.614655 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:40.614959 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:40.616037 (+ 1078us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:31:40.616042 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:40.616135 (+ 93us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:31:40.616309 (+ 174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:40.617295 (+ 986us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:40.617324 (+ 29us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:40.617335 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 00:31:40.617628 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 00:31:40.618235 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":368,"threads_started":1}
I20260811 00:31:40.621876 25452 heartbeater.cc:347] Connected to a master server at 127.2.120.62:32801
I20260811 00:31:40.622283 25452 heartbeater.cc:464] Registering TS with master...
I20260811 00:31:40.623236 25452 heartbeater.cc:511] Master 127.2.120.62:32801 requested a full tablet report, sending...
I20260811 00:31:40.626147 25241 ts_manager.cc:194] Registered new tserver with Master: 8dfd44fa95df4a3f89f36677399a18d7 (127.2.120.1:33873)
I20260811 00:31:40.628695 25241 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.120.1:51867
W20260811 00:31:41.021974 25457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:41.022326 25457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:41.022419 25457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:41.030059 25457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:41.030197 25457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:41.030311 25457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.2
I20260811 00:31:41.041646 25457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:33253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.120.2
--webserver_port=0
--tserver_master_addrs=127.2.120.62:32801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.2
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:41.043041 25457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:41.045089 25457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:41.060809 25467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:41.060873 25464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:41.061731 25465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:41.062217 25457 server_base.cc:1034] running on GCE node
I20260811 00:31:41.063342 25457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:41.064699 25457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:41.065977 25457 hybrid_clock.cc:648] HybridClock initialized: now 1786408301065919 us; error 55 us; skew 500 ppm
I20260811 00:31:41.066357 25457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:31:41.069164 25457 webserver.cc:460] Webserver started at http://127.2.120.2:46707/ using document root <none> and password file <none>
I20260811 00:31:41.069977 25457 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:41.070122 25457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:41.070484 25457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:41.073698 25457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-1/data/instance:
uuid: "25c686304aed49b089210f258d982928"
format_stamp: "Formatted at 2026-08-11 00:31:41 on dist-test-slave-fd7l"
server_key: "fe45cb7bb081710d1265535474a4a7d6"
server_key_iv: "9c96dfb5f28ed976481e95e2861576c8"
server_key_version: "encryptionkey@0"
I20260811 00:31:41.074582 25457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance:
uuid: "25c686304aed49b089210f258d982928"
format_stamp: "Formatted at 2026-08-11 00:31:41 on dist-test-slave-fd7l"
server_key: "fe45cb7bb081710d1265535474a4a7d6"
server_key_iv: "9c96dfb5f28ed976481e95e2861576c8"
server_key_version: "encryptionkey@0"
I20260811 00:31:41.080984 25457 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 00:31:41.085628 25474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:41.087136 25457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:31:41.087364 25457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-1/wal
uuid: "25c686304aed49b089210f258d982928"
format_stamp: "Formatted at 2026-08-11 00:31:41 on dist-test-slave-fd7l"
server_key: "fe45cb7bb081710d1265535474a4a7d6"
server_key_iv: "9c96dfb5f28ed976481e95e2861576c8"
server_key_version: "encryptionkey@0"
I20260811 00:31:41.087658 25457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:41.107859 25457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:41.109613 25457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:41.110096 25457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:41.111743 25457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:41.114182 25457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:41.114328 25457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:41.114488 25457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:41.114590 25457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:41.154168 25457 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.2:45621
I20260811 00:31:41.154271 25619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.2:45621 every 8 connection(s)
I20260811 00:31:41.155666 25457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-1/data/info.pb
I20260811 00:31:41.160306 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 25457
I20260811 00:31:41.160549 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-1/wal/instance
I20260811 00:31:41.161195 2528 external_mini_cluster.cc:1428] Setting key d46fe1519aab5b27384f797e5e8e8dfc
I20260811 00:31:41.164812 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.120.3:0
--local_ip_for_outbound_sockets=127.2.120.3
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:32801
--builtin_ntp_servers=127.2.120.20:33253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 00:31:41.172686 25622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:41.158707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.2:34887 (local address 127.2.120.62:32801)
0811 00:31:41.159223 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:41.159228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:41.160846 (+ 1618us) server_negotiation.cc:407] Connection header received
0811 00:31:41.163263 (+ 2417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:41.163270 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:41.163359 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:41.163490 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:41.165345 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.166108 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.169110 (+ 3002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.169596 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.169730 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:41.171736 (+ 2006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:41.171742 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:41.171746 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:41.171771 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:41.171829 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:41.171842 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:41.171917 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:41.172221 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 00:31:41.172319 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":283,"threads_started":1}
I20260811 00:31:41.174115 25623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:41.159165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:32801 (local address 127.2.120.2:34887)
0811 00:31:41.160552 (+ 1387us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:41.160618 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:41.162891 (+ 2273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:41.163865 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:41.163880 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:41.164040 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:41.165121 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:41.165157 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.166295 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.166304 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:41.168877 (+ 2573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:41.168900 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.169899 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.169910 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:41.170301 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:41.171218 (+ 917us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:31:41.171223 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:41.171291 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:31:41.171446 (+ 155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:41.171952 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:41.171965 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:41.171976 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 00:31:41.172188 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 00:31:41.172761 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":344,"threads_started":1}
I20260811 00:31:41.176015 25620 heartbeater.cc:347] Connected to a master server at 127.2.120.62:32801
I20260811 00:31:41.176448 25620 heartbeater.cc:464] Registering TS with master...
I20260811 00:31:41.177321 25620 heartbeater.cc:511] Master 127.2.120.62:32801 requested a full tablet report, sending...
I20260811 00:31:41.179175 25241 ts_manager.cc:194] Registered new tserver with Master: 25c686304aed49b089210f258d982928 (127.2.120.2:45621)
I20260811 00:31:41.180217 25241 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.120.2:34887
W20260811 00:31:41.566064 25624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:41.566401 25624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:41.566493 25624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:41.574638 25624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:41.574817 25624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:41.575007 25624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.120.3
I20260811 00:31:41.585675 25624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:33253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.120.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.120.3
--webserver_port=0
--tserver_master_addrs=127.2.120.62:32801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.120.3
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:41.587283 25624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:41.589637 25624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:41.607771 25632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:41.608023 25629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:41.608237 25630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:41.609025 25624 server_base.cc:1034] running on GCE node
I20260811 00:31:41.609774 25624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:41.610980 25624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:41.612393 25624 hybrid_clock.cc:648] HybridClock initialized: now 1786408301612292 us; error 51 us; skew 500 ppm
I20260811 00:31:41.613137 25624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:31:41.617192 25624 webserver.cc:460] Webserver started at http://127.2.120.3:46791/ using document root <none> and password file <none>
I20260811 00:31:41.618285 25624 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:41.618703 25624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:41.619248 25624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:41.622890 25624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-2/data/instance:
uuid: "5320833fdd41498397fba07e6ebc89f0"
format_stamp: "Formatted at 2026-08-11 00:31:41 on dist-test-slave-fd7l"
server_key: "8e1a5e37ca6e279bd166f52dab687063"
server_key_iv: "564182dece896a2180214c72158124bd"
server_key_version: "encryptionkey@0"
I20260811 00:31:41.623873 25624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance:
uuid: "5320833fdd41498397fba07e6ebc89f0"
format_stamp: "Formatted at 2026-08-11 00:31:41 on dist-test-slave-fd7l"
server_key: "8e1a5e37ca6e279bd166f52dab687063"
server_key_iv: "564182dece896a2180214c72158124bd"
server_key_version: "encryptionkey@0"
I20260811 00:31:41.630601 25624 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260811 00:31:41.632766 25452 heartbeater.cc:503] Master 127.2.120.62:32801 was elected leader, sending a full tablet report...
I20260811 00:31:41.635059 25638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:41.636674 25624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:31:41.636816 25624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-2/wal
uuid: "5320833fdd41498397fba07e6ebc89f0"
format_stamp: "Formatted at 2026-08-11 00:31:41 on dist-test-slave-fd7l"
server_key: "8e1a5e37ca6e279bd166f52dab687063"
server_key_iv: "564182dece896a2180214c72158124bd"
server_key_version: "encryptionkey@0"
I20260811 00:31:41.637027 25624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:41.661568 25624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:41.662528 25624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:41.662796 25624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:41.663903 25624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:31:41.665817 25624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:31:41.665910 25624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:41.665987 25624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:31:41.666052 25624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:41.705265 25624 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.3:46851
I20260811 00:31:41.705374 25770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.3:46851 every 8 connection(s)
I20260811 00:31:41.706821 25624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-2/data/info.pb
I20260811 00:31:41.716663 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 25624
I20260811 00:31:41.716843 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-2/wal/instance
I20260811 00:31:41.717588 2528 external_mini_cluster.cc:1428] Setting key a430741de0440db1fb4cdf0781425a49
I20260811 00:31:41.723426 25773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:41.710262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:55957 (local address 127.2.120.62:32801)
0811 00:31:41.710945 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:41.710951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:41.711969 (+ 1018us) server_negotiation.cc:407] Connection header received
0811 00:31:41.714081 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:41.714096 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:41.714185 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:41.714304 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:41.715625 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.716906 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.719591 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.720226 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.720383 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:41.722479 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:41.722485 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:41.722489 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:41.722513 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:41.722564 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:41.722579 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:41.722663 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:41.722971 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 00:31:41.723077 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":391,"threads_started":1}
I20260811 00:31:41.724586 25774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:41.710473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:32801 (local address 127.2.120.3:55957)
0811 00:31:41.711710 (+ 1237us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:41.711770 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:41.713665 (+ 1895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:41.714563 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:41.714574 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:41.714674 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:31:41.715315 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:41.715335 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.717226 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.717236 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:41.719333 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:41.719356 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.720580 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.720590 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:41.720861 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:41.721956 (+ 1095us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:31:41.721959 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:31:41.722023 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:31:41.722143 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:31:41.722683 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:31:41.722693 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:31:41.722700 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:31:41.722877 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 00:31:41.723254 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":394,"threads_started":1}
I20260811 00:31:41.726703 25771 heartbeater.cc:347] Connected to a master server at 127.2.120.62:32801
I20260811 00:31:41.727087 25771 heartbeater.cc:464] Registering TS with master...
I20260811 00:31:41.728001 25771 heartbeater.cc:511] Master 127.2.120.62:32801 requested a full tablet report, sending...
I20260811 00:31:41.729981 25241 ts_manager.cc:194] Registered new tserver with Master: 5320833fdd41498397fba07e6ebc89f0 (127.2.120.3:46851)
I20260811 00:31:41.730919 25241 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.120.3:55957
I20260811 00:31:41.735792 2528 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:31:41.749611 25773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:41.741458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40112 (local address 127.2.120.62:32801)
0811 00:31:41.741784 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:41.741823 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:41.742307 (+ 484us) server_negotiation.cc:407] Connection header received
0811 00:31:41.742577 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:41.742582 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:41.742662 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:41.742763 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:41.743350 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.744219 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.746006 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.746493 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.746643 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:41.748483 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:41.748490 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:41.748494 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:41.748624 (+ 130us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:41.748674 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:41.748691 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:41.748805 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:41.749084 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 00:31:41.749194 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260811 00:31:41.755496 25241 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:31:41.765172 25241 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:31:41.795395 25786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:41.780308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:33873 (local address 127.0.0.1:34496)
0811 00:31:41.781461 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:41.781557 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:41.781813 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:41.785509 (+ 3696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:41.785522 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:41.785563 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:41.786029 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:41.786046 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.789054 (+ 3008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.789060 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:41.791791 (+ 2731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:41.791811 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.794231 (+ 2420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.794238 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:41.794454 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:41.794515 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 00:31:41.794686 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 00:31:41.794929 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"mutex_wait_us":1,"thread_start_us":543,"threads_started":1}
I20260811 00:31:41.796027 25789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:41.781411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35804 (local address 127.2.120.3:46851)
0811 00:31:41.786318 (+ 4907us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:41.786328 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:41.786366 (+ 38us) server_negotiation.cc:407] Connection header received
0811 00:31:41.786495 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:41.786504 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:41.786754 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:41.788081 (+ 1327us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:41.788614 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.790610 (+ 1996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.793384 (+ 2774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.794677 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.794829 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:41.795072 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:41.795366 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 00:31:41.795598 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4718,"thread_start_us":4544,"threads_started":1}
I20260811 00:31:41.796391 25787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:41.780707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.3:46851 (local address 127.0.0.1:35804)
0811 00:31:41.783711 (+ 3004us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:41.783751 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:41.783925 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:41.787028 (+ 3103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:41.787037 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:41.787077 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:41.787595 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:41.787610 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.790836 (+ 3226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.790847 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:41.793171 (+ 2324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:41.793184 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.794889 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.794895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:41.795151 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:41.795185 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 00:31:41.795287 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 00:31:41.795446 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2795,"mutex_wait_us":130,"thread_start_us":2524,"threads_started":1}
I20260811 00:31:41.796638 25788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:41.780692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34496 (local address 127.2.120.1:33873)
0811 00:31:41.784618 (+ 3926us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:41.784628 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:41.784667 (+ 39us) server_negotiation.cc:407] Connection header received
0811 00:31:41.784806 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:41.784815 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:41.785170 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:41.786291 (+ 1121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:41.786605 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.788851 (+ 2246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.792176 (+ 3325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.793967 (+ 1791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.794146 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:41.794500 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:41.795992 (+ 1492us) server_negotiation.cc:299] Negotiation successful
0811 00:31:41.796238 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3675,"thread_start_us":3524,"threads_started":1}
I20260811 00:31:41.797123 25790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:41.783518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:45621 (local address 127.0.0.1:54180)
0811 00:31:41.785468 (+ 1950us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:41.785495 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:41.785638 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:41.786422 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:41.786430 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:41.786464 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:41.787478 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:41.787493 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.789685 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.789691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:41.792432 (+ 2741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:41.792448 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.794666 (+ 2218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.794673 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:41.794892 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:41.796564 (+ 1672us) client_negotiation.cc:769] Sending connection context
0811 00:31:41.796657 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 00:31:41.796816 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1789,"mutex_wait_us":1605,"spinlock_wait_cycles":6016,"thread_start_us":1623,"threads_started":1}
I20260811 00:31:41.797997 25791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:41.784248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54180 (local address 127.2.120.2:45621)
0811 00:31:41.784937 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:41.784945 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:41.785634 (+ 689us) server_negotiation.cc:407] Connection header received
0811 00:31:41.785856 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:41.785869 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:41.786155 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:41.787741 (+ 1586us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:41.787985 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.789515 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.792701 (+ 3186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.794275 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.794427 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:41.794681 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:41.797519 (+ 2838us) server_negotiation.cc:299] Negotiation successful
0811 00:31:41.797684 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":404,"threads_started":1}
I20260811 00:31:41.800998 25695 tablet_service.cc:1467] Processing CreateTablet for tablet 627609cbcc0f4e9c9750ea986d5c2a73 (DEFAULT_TABLE table=test-table [id=ea54d8c6fb51417b96c05a1e1757828e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:31:41.800992 25379 tablet_service.cc:1467] Processing CreateTablet for tablet 627609cbcc0f4e9c9750ea986d5c2a73 (DEFAULT_TABLE table=test-table [id=ea54d8c6fb51417b96c05a1e1757828e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:31:41.801519 25538 tablet_service.cc:1467] Processing CreateTablet for tablet 627609cbcc0f4e9c9750ea986d5c2a73 (DEFAULT_TABLE table=test-table [id=ea54d8c6fb51417b96c05a1e1757828e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:31:41.802527 25379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627609cbcc0f4e9c9750ea986d5c2a73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:41.802596 25695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627609cbcc0f4e9c9750ea986d5c2a73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:41.802874 25538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627609cbcc0f4e9c9750ea986d5c2a73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:41.817368 25792 tablet_bootstrap.cc:492] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7: Bootstrap starting.
I20260811 00:31:41.821193 25792 tablet_bootstrap.cc:654] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:41.821415 25793 tablet_bootstrap.cc:492] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928: Bootstrap starting.
I20260811 00:31:41.822881 25792 log.cc:824] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:41.825661 25793 tablet_bootstrap.cc:654] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:41.825937 25794 tablet_bootstrap.cc:492] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0: Bootstrap starting.
I20260811 00:31:41.826903 25792 tablet_bootstrap.cc:492] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7: No bootstrap required, opened a new log
I20260811 00:31:41.827042 25793 log.cc:824] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:41.827286 25792 ts_tablet_manager.cc:1397] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 00:31:41.829736 25794 tablet_bootstrap.cc:654] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:41.830668 25793 tablet_bootstrap.cc:492] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928: No bootstrap required, opened a new log
I20260811 00:31:41.830960 25793 ts_tablet_manager.cc:1397] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260811 00:31:41.831055 25794 log.cc:824] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:41.834483 25794 tablet_bootstrap.cc:492] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0: No bootstrap required, opened a new log
I20260811 00:31:41.834784 25794 ts_tablet_manager.cc:1397] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 00:31:41.836774 25792 raft_consensus.cc:348] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfd44fa95df4a3f89f36677399a18d7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33873 } } peers { permanent_uuid: "5320833fdd41498397fba07e6ebc89f0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46851 } } peers { permanent_uuid: "25c686304aed49b089210f258d982928" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45621 } }
I20260811 00:31:41.837347 25792 raft_consensus.cc:374] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:41.837460 25792 raft_consensus.cc:729] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dfd44fa95df4a3f89f36677399a18d7, State: Initialized, Role: FOLLOWER
I20260811 00:31:41.838310 25792 consensus_queue.cc:260] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfd44fa95df4a3f89f36677399a18d7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33873 } } peers { permanent_uuid: "5320833fdd41498397fba07e6ebc89f0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46851 } } peers { permanent_uuid: "25c686304aed49b089210f258d982928" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45621 } }
I20260811 00:31:41.840360 25792 ts_tablet_manager.cc:1428] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260811 00:31:41.841867 25793 raft_consensus.cc:348] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfd44fa95df4a3f89f36677399a18d7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33873 } } peers { permanent_uuid: "5320833fdd41498397fba07e6ebc89f0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46851 } } peers { permanent_uuid: "25c686304aed49b089210f258d982928" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45621 } }
I20260811 00:31:41.842407 25793 raft_consensus.cc:374] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:41.842502 25793 raft_consensus.cc:729] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25c686304aed49b089210f258d982928, State: Initialized, Role: FOLLOWER
I20260811 00:31:41.843185 25793 consensus_queue.cc:260] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfd44fa95df4a3f89f36677399a18d7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33873 } } peers { permanent_uuid: "5320833fdd41498397fba07e6ebc89f0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46851 } } peers { permanent_uuid: "25c686304aed49b089210f258d982928" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45621 } }
I20260811 00:31:41.844794 25620 heartbeater.cc:503] Master 127.2.120.62:32801 was elected leader, sending a full tablet report...
I20260811 00:31:41.845350 25793 ts_tablet_manager.cc:1428] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
I20260811 00:31:41.848699 25794 raft_consensus.cc:348] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfd44fa95df4a3f89f36677399a18d7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33873 } } peers { permanent_uuid: "5320833fdd41498397fba07e6ebc89f0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46851 } } peers { permanent_uuid: "25c686304aed49b089210f258d982928" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45621 } }
I20260811 00:31:41.849268 25794 raft_consensus.cc:374] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:41.849381 25794 raft_consensus.cc:729] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5320833fdd41498397fba07e6ebc89f0, State: Initialized, Role: FOLLOWER
I20260811 00:31:41.850301 25794 consensus_queue.cc:260] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfd44fa95df4a3f89f36677399a18d7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33873 } } peers { permanent_uuid: "5320833fdd41498397fba07e6ebc89f0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46851 } } peers { permanent_uuid: "25c686304aed49b089210f258d982928" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45621 } }
I20260811 00:31:41.851850 25771 heartbeater.cc:503] Master 127.2.120.62:32801 was elected leader, sending a full tablet report...
I20260811 00:31:41.852373 25794 ts_tablet_manager.cc:1428] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
W20260811 00:31:41.855916 25453 tablet.cc:2369] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:31:41.908209 25621 tablet.cc:2369] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:31:41.958885 25772 tablet.cc:2369] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:31:41.965834 25800 raft_consensus.cc:482] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:31:41.966228 25800 raft_consensus.cc:504] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfd44fa95df4a3f89f36677399a18d7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33873 } } peers { permanent_uuid: "5320833fdd41498397fba07e6ebc89f0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46851 } } peers { permanent_uuid: "25c686304aed49b089210f258d982928" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45621 } }
I20260811 00:31:41.968173 25800 leader_election.cc:288] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dfd44fa95df4a3f89f36677399a18d7 (127.2.120.1:33873), 25c686304aed49b089210f258d982928 (127.2.120.2:45621)
I20260811 00:31:41.976279 25791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:41.968654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:50879 (local address 127.2.120.2:45621)
0811 00:31:41.968878 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:41.968885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:41.969130 (+ 245us) server_negotiation.cc:407] Connection header received
0811 00:31:41.969407 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:41.969415 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:41.969528 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:41.969616 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:41.970232 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.971212 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.973835 (+ 2623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.975207 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.975391 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:41.975533 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:41.975822 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 00:31:41.975995 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 00:31:41.976320 25774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:41.968645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.2:45621 (local address 127.2.120.3:50879)
0811 00:31:41.968988 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:41.969008 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:41.969211 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:41.969748 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:41.969753 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:41.969793 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:41.970041 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:41.970065 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.971369 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.971373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:41.973586 (+ 2213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:41.973606 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.975460 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.975466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:41.975604 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:41.975632 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 00:31:41.975788 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 00:31:41.975960 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":59}
I20260811 00:31:41.976737 25788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:41.968795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.120.3:45149 (local address 127.2.120.1:33873)
0811 00:31:41.969025 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:41.969035 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:41.969754 (+ 719us) server_negotiation.cc:407] Connection header received
0811 00:31:41.970106 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:41.970110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:41.970232 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:41.970343 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:31:41.970909 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.971743 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.974183 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.975647 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.975819 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:41.975922 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:41.976262 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 00:31:41.976438 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 00:31:41.977140 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:41.968683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:33873 (local address 127.2.120.3:45149)
0811 00:31:41.969654 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:41.969674 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:31:41.969899 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:31:41.970384 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:31:41.970388 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:31:41.970410 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:31:41.970728 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:41.970736 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.971909 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.971914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:41.973960 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:31:41.973971 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:31:41.975865 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:31:41.975871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:31:41.976064 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:41.976098 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 00:31:41.976197 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 00:31:41.976335 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"mutex_wait_us":4,"spinlock_wait_cycles":46336,"thread_start_us":529,"threads_started":1}
I20260811 00:31:41.977550 25565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627609cbcc0f4e9c9750ea986d5c2a73" candidate_uuid: "5320833fdd41498397fba07e6ebc89f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25c686304aed49b089210f258d982928" is_pre_election: true
I20260811 00:31:41.978237 25565 raft_consensus.cc:2454] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5320833fdd41498397fba07e6ebc89f0 in term 0.
I20260811 00:31:41.978475 25402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627609cbcc0f4e9c9750ea986d5c2a73" candidate_uuid: "5320833fdd41498397fba07e6ebc89f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dfd44fa95df4a3f89f36677399a18d7" is_pre_election: true
I20260811 00:31:41.979101 25402 raft_consensus.cc:2454] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5320833fdd41498397fba07e6ebc89f0 in term 0.
I20260811 00:31:41.979137 25642 leader_election.cc:302] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25c686304aed49b089210f258d982928, 5320833fdd41498397fba07e6ebc89f0; no voters:
I20260811 00:31:41.979705 25800 raft_consensus.cc:2781] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:31:41.979888 25800 raft_consensus.cc:482] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:31:41.980049 25800 raft_consensus.cc:3037] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:41.982797 25800 raft_consensus.cc:504] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfd44fa95df4a3f89f36677399a18d7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33873 } } peers { permanent_uuid: "5320833fdd41498397fba07e6ebc89f0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46851 } } peers { permanent_uuid: "25c686304aed49b089210f258d982928" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45621 } }
I20260811 00:31:41.983567 25800 leader_election.cc:288] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [CANDIDATE]: Term 1 election: Requested vote from peers 8dfd44fa95df4a3f89f36677399a18d7 (127.2.120.1:33873), 25c686304aed49b089210f258d982928 (127.2.120.2:45621)
I20260811 00:31:41.984215 25402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627609cbcc0f4e9c9750ea986d5c2a73" candidate_uuid: "5320833fdd41498397fba07e6ebc89f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dfd44fa95df4a3f89f36677399a18d7"
I20260811 00:31:41.984220 25565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627609cbcc0f4e9c9750ea986d5c2a73" candidate_uuid: "5320833fdd41498397fba07e6ebc89f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25c686304aed49b089210f258d982928"
I20260811 00:31:41.984472 25565 raft_consensus.cc:3037] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:41.984468 25402 raft_consensus.cc:3037] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:41.986868 25565 raft_consensus.cc:2454] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5320833fdd41498397fba07e6ebc89f0 in term 1.
I20260811 00:31:41.986965 25402 raft_consensus.cc:2454] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5320833fdd41498397fba07e6ebc89f0 in term 1.
I20260811 00:31:41.987483 25642 leader_election.cc:302] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25c686304aed49b089210f258d982928, 5320833fdd41498397fba07e6ebc89f0; no voters:
I20260811 00:31:41.987844 25800 raft_consensus.cc:2781] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:41.988366 25800 raft_consensus.cc:686] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [term 1 LEADER]: Becoming Leader. State: Replica: 5320833fdd41498397fba07e6ebc89f0, State: Running, Role: LEADER
I20260811 00:31:41.989020 25800 consensus_queue.cc:237] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfd44fa95df4a3f89f36677399a18d7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33873 } } peers { permanent_uuid: "5320833fdd41498397fba07e6ebc89f0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46851 } } peers { permanent_uuid: "25c686304aed49b089210f258d982928" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45621 } }
I20260811 00:31:41.994184 25240 catalog_manager.cc:5495] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5320833fdd41498397fba07e6ebc89f0 (127.2.120.3). New cstate: current_term: 1 leader_uuid: "5320833fdd41498397fba07e6ebc89f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfd44fa95df4a3f89f36677399a18d7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5320833fdd41498397fba07e6ebc89f0" member_type: VOTER last_known_addr { host: "127.2.120.3" port: 46851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25c686304aed49b089210f258d982928" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45621 } health_report { overall_health: UNKNOWN } } }
I20260811 00:31:42.040264 25789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:42.034185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35814 (local address 127.2.120.3:46851)
0811 00:31:42.034436 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:42.034442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:42.034464 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:31:42.034711 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:42.034715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:42.034804 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:42.034987 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:31:42.035388 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:42.036127 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:42.037731 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:42.038181 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:42.038378 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:42.039042 (+ 664us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:31:42.039612 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:31:42.039784 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:42.039889 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 00:31:42.039982 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 00:31:42.050935 25402 raft_consensus.cc:1261] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7 [term 1 FOLLOWER]: Refusing update from remote peer 5320833fdd41498397fba07e6ebc89f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:31:42.050935 25565 raft_consensus.cc:1261] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928 [term 1 FOLLOWER]: Refusing update from remote peer 5320833fdd41498397fba07e6ebc89f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:31:42.052299 25800 consensus_queue.cc:1035] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25c686304aed49b089210f258d982928" member_type: VOTER last_known_addr { host: "127.2.120.2" port: 45621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:31:42.052772 25803 consensus_queue.cc:1035] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dfd44fa95df4a3f89f36677399a18d7" member_type: VOTER last_known_addr { host: "127.2.120.1" port: 33873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:31:42.074011 25809 mvcc.cc:205] Tried to move back new op lower bound from 7317128405185720320 to 7317128404951490560. Current Snapshot: MvccSnapshot[applied={T|T < 7317128405185720320}]
I20260811 00:31:42.084457 25241 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:31:42.084856 25241 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:31:42.089634 25241 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44b453cdca574943ac645f45641d9d1f: Sending DeleteTablet for 3 replicas of tablet 627609cbcc0f4e9c9750ea986d5c2a73
I20260811 00:31:42.091221 25379 tablet_service.cc:1514] Processing DeleteTablet for tablet 627609cbcc0f4e9c9750ea986d5c2a73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:31:42 UTC) from {username='slave'} at 127.0.0.1:34496
I20260811 00:31:42.091219 25538 tablet_service.cc:1514] Processing DeleteTablet for tablet 627609cbcc0f4e9c9750ea986d5c2a73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:31:42 UTC) from {username='slave'} at 127.0.0.1:54180
I20260811 00:31:42.091543 25695 tablet_service.cc:1514] Processing DeleteTablet for tablet 627609cbcc0f4e9c9750ea986d5c2a73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:31:42 UTC) from {username='slave'} at 127.0.0.1:35804
I20260811 00:31:42.099258 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 25306
I20260811 00:31:42.106979 25821 tablet_replica.cc:330] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0: stopping tablet replica
I20260811 00:31:42.107616 25821 raft_consensus.cc:2229] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:31:42.109561 25818 tablet_replica.cc:330] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7: stopping tablet replica
I20260811 00:31:42.110122 25818 raft_consensus.cc:2229] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:31:42.110752 25818 raft_consensus.cc:2258] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:31:42.108474 25821 raft_consensus.cc:2258] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:31:42.113517 25818 ts_tablet_manager.cc:1905] T 627609cbcc0f4e9c9750ea986d5c2a73 P 8dfd44fa95df4a3f89f36677399a18d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:31:42.113958 25821 ts_tablet_manager.cc:1905] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:31:42.117707 25788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:42.100817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34506 (local address 127.2.120.1:33873)
0811 00:31:42.101053 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:42.101060 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:42.106855 (+ 5795us) server_negotiation.cc:407] Connection header received
0811 00:31:42.107340 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:42.107347 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:42.107472 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:42.107578 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:42.108560 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:42.110285 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:42.112865 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:42.113632 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:42.113743 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:42.115517 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:42.115532 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:42.115551 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:42.116394 (+ 843us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:42.116463 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:42.116489 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:42.116624 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:42.116951 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 00:31:42.117145 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 00:31:42.119648 25447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:42.120393 25820 tablet_replica.cc:330] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928: stopping tablet replica
I20260811 00:31:42.120952 25820 raft_consensus.cc:2229] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:31:42.121549 25820 raft_consensus.cc:2258] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:31:42.124266 25820 ts_tablet_manager.cc:1905] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:31:42.128760 25821 ts_tablet_manager.cc:1918] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:31:42.128991 25821 log.cc:1196] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-2/wal/wals/627609cbcc0f4e9c9750ea986d5c2a73
I20260811 00:31:42.129629 25821 ts_tablet_manager.cc:1939] T 627609cbcc0f4e9c9750ea986d5c2a73 P 5320833fdd41498397fba07e6ebc89f0: Deleting consensus metadata
I20260811 00:31:42.131883 25224 catalog_manager.cc:4841] TS 5320833fdd41498397fba07e6ebc89f0 (127.2.120.3:46851): tablet 627609cbcc0f4e9c9750ea986d5c2a73 (table test-table [id=ea54d8c6fb51417b96c05a1e1757828e]) successfully deleted
I20260811 00:31:42.132335 25820 ts_tablet_manager.cc:1918] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:31:42.132577 25820 log.cc:1196] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928: Deleting WAL directory at /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407939260739-2528-0/minicluster-data/ts-1/wal/wals/627609cbcc0f4e9c9750ea986d5c2a73
I20260811 00:31:42.133195 25820 ts_tablet_manager.cc:1939] T 627609cbcc0f4e9c9750ea986d5c2a73 P 25c686304aed49b089210f258d982928: Deleting consensus metadata
I20260811 00:31:42.135468 25227 catalog_manager.cc:4841] TS 25c686304aed49b089210f258d982928 (127.2.120.2:45621): tablet 627609cbcc0f4e9c9750ea986d5c2a73 (table test-table [id=ea54d8c6fb51417b96c05a1e1757828e]) successfully deleted
I20260811 00:31:42.221920 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 25306
I20260811 00:31:42.247117 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 25457
W20260811 00:31:42.247017 25226 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.120.1:33873 (error 108)
I20260811 00:31:42.249393 25790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:42.248859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:33873 (local address 127.0.0.1:34516)
0811 00:31:42.249055 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:42.249191 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260811 00:31:42.250895 25226 catalog_manager.cc:4584] TS 8dfd44fa95df4a3f89f36677399a18d7 (127.2.120.1:33873): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 627609cbcc0f4e9c9750ea986d5c2a73: Network error: Client connection negotiation failed: client connection to 127.2.120.1:33873: connect: Connection refused (error 111)
I20260811 00:31:42.256062 25791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:42.248461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54184 (local address 127.2.120.2:45621)
0811 00:31:42.248696 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:42.248702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:42.248726 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:31:42.248803 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:42.248808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:42.248939 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:42.249080 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:42.249672 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:42.250646 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:42.252659 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:42.253083 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:42.253204 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:42.254472 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:42.254482 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:42.254498 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:42.255194 (+ 696us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:42.255246 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:42.255268 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:42.255370 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:42.255550 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 00:31:42.255654 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260811 00:31:42.257028 25615 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:42.305173 25790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:42.304582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:33873 (local address 127.0.0.1:34524)
0811 00:31:42.304850 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:42.304979 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 00:31:42.306517 25790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:42.306031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:33873 (local address 127.0.0.1:34536)
0811 00:31:42.306219 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:42.306341 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 00:31:42.352174 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 25457
I20260811 00:31:42.367468 25790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:42.366715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:33873 (local address 127.0.0.1:34544)
0811 00:31:42.367007 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:42.367202 (+ 195us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 00:31:42.369014 25790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:42.368489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:33873 (local address 127.0.0.1:34558)
0811 00:31:42.368711 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:42.368832 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 00:31:42.377233 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 25624
I20260811 00:31:42.386194 25789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:42.378238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35824 (local address 127.2.120.3:46851)
0811 00:31:42.378414 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:42.378423 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:42.378542 (+ 119us) server_negotiation.cc:407] Connection header received
0811 00:31:42.378784 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:42.378789 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:42.378919 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:42.379019 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:42.379622 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:42.380759 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:42.382734 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:42.383204 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:42.383307 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:42.384690 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:42.384699 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:42.384714 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:42.385344 (+ 630us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:42.385399 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:42.385420 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:42.385506 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:42.385707 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 00:31:42.385798 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:31:42.387310 25766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:42.449967 25790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:42.449390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:33873 (local address 127.0.0.1:34572)
0811 00:31:42.449652 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:42.449759 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260811 00:31:42.451211 25790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:42.450767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:33873 (local address 127.0.0.1:34578)
0811 00:31:42.451008 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:42.451085 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 00:31:42.519191 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 25624
I20260811 00:31:42.545090 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 25205
I20260811 00:31:42.546406 25275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:42.660951 25790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:42.660285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.1:33873 (local address 127.0.0.1:34592)
0811 00:31:42.660567 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 00:31:42.660709 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.1:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 00:31:42.661125 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 25205
2026-08-11T00:31:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3370 ms)
[----------] 6 tests from EncryptionPolicyTest (26525 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:31:42 dist-test-slave-fd7l krb5kdc[25831](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:31:42 dist-test-slave-fd7l krb5kdc[25831](info): set up 2 sockets
Aug 11 00:31:42 dist-test-slave-fd7l krb5kdc[25831](info): commencing operation
krb5kdc: starting...
W20260811 00:31:43.979072 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:31:44 dist-test-slave-fd7l krb5kdc[25831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:31:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:31:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:31:44.337116 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36075
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:31:44.735162 25847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:44.735538 25847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:44.735711 25847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:44.743844 25847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:31:44.744005 25847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:44.744093 25847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:44.744206 25847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:31:44.744264 25847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:31:44.755849 25847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36075
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:44.757732 25847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:44.759620 25847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:44.773905 25856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:44.773931 25858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:44.773980 25855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:44.775002 25847 server_base.cc:1034] running on GCE node
I20260811 00:31:44.775961 25847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:44.777801 25847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:44.779054 25847 hybrid_clock.cc:648] HybridClock initialized: now 1786408304779003 us; error 55 us; skew 500 ppm
I20260811 00:31:44.779438 25847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:44 dist-test-slave-fd7l krb5kdc[25831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408304, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:44.783020 25847 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:31:44.784992 25847 webserver.cc:460] Webserver started at http://127.0.0.1:33699/ using document root <none> and password file <none>
I20260811 00:31:44.785812 25847 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:44.785956 25847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:44.786324 25847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:44.789801 25847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "a2eccc660cec439eb43493ac3c96520b"
format_stamp: "Formatted at 2026-08-11 00:31:44 on dist-test-slave-fd7l"
server_key: "683cf76ade01f750f1a16e7378990392"
server_key_iv: "93ac0a3267f4eca9f5e3470198d21486"
server_key_version: "encryptionkey@0"
I20260811 00:31:44.790733 25847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "a2eccc660cec439eb43493ac3c96520b"
format_stamp: "Formatted at 2026-08-11 00:31:44 on dist-test-slave-fd7l"
server_key: "683cf76ade01f750f1a16e7378990392"
server_key_iv: "93ac0a3267f4eca9f5e3470198d21486"
server_key_version: "encryptionkey@0"
I20260811 00:31:44.796664 25847 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 00:31:44.800666 25865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:44.802206 25847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 00:31:44.802420 25847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "a2eccc660cec439eb43493ac3c96520b"
format_stamp: "Formatted at 2026-08-11 00:31:44 on dist-test-slave-fd7l"
server_key: "683cf76ade01f750f1a16e7378990392"
server_key_iv: "93ac0a3267f4eca9f5e3470198d21486"
server_key_version: "encryptionkey@0"
I20260811 00:31:44.802664 25847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:44.823153 25847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:44.828339 25847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:44.828729 25847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:44.850142 25847 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36075
I20260811 00:31:44.850193 25925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36075 every 8 connection(s)
I20260811 00:31:44.851730 25847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:31:44.852375 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 25847
I20260811 00:31:44.852698 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:31:44.853535 2528 external_mini_cluster.cc:1428] Setting key 4216dd40f42bdd7adb8b445952b329b8
I20260811 00:31:44.856501 25926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 00:31:44 dist-test-slave-fd7l krb5kdc[25831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 00:31:44.867808 25926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b: Bootstrap starting.
I20260811 00:31:44.871455 25926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:44.873122 25926 log.cc:824] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:44.877413 25926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b: No bootstrap required, opened a new log
I20260811 00:31:44.880386 25929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:44.855614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39883 (local address 127.0.0.1:36075)
0811 00:31:44.856396 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:44.856409 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:44.856453 (+ 44us) server_negotiation.cc:407] Connection header received
0811 00:31:44.858264 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:44.858287 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:44.858680 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:44.859101 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:44.860607 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:44.861975 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:44.863322 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:44.864036 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:44.864265 (+ 229us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:44.869766 (+ 5501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:44.869776 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:44.869793 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:44.869838 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:44.874091 (+ 4253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:44.875429 (+ 1338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:44.875442 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:44.875451 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:44.875711 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:44.876304 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:44.876312 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:44.876318 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:44.877174 (+ 856us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:44.877586 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:44.878337 (+ 751us) server_negotiation.cc:299] Negotiation successful
0811 00:31:44.878783 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":356,"threads_started":1}
I20260811 00:31:44.885768 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:31:44.888350 25926 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2eccc660cec439eb43493ac3c96520b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36075 } }
I20260811 00:31:44.888985 25926 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:44.889129 25926 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2eccc660cec439eb43493ac3c96520b, State: Initialized, Role: FOLLOWER
I20260811 00:31:44.889968 25926 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2eccc660cec439eb43493ac3c96520b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36075 } }
I20260811 00:31:44.890245 25926 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:31:44.890415 25926 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:31:44.890631 25926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:44.893747 25926 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2eccc660cec439eb43493ac3c96520b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36075 } }
I20260811 00:31:44.894531 25926 leader_election.cc:302] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2eccc660cec439eb43493ac3c96520b; no voters:
I20260811 00:31:44.895368 25926 leader_election.cc:288] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:31:44.895527 25937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:44.897042 25937 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b [term 1 LEADER]: Becoming Leader. State: Replica: a2eccc660cec439eb43493ac3c96520b, State: Running, Role: LEADER
I20260811 00:31:44.897821 25937 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2eccc660cec439eb43493ac3c96520b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36075 } }
I20260811 00:31:44.898416 25926 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:31:44.901232 25939 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2eccc660cec439eb43493ac3c96520b. Latest consensus state: current_term: 1 leader_uuid: "a2eccc660cec439eb43493ac3c96520b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2eccc660cec439eb43493ac3c96520b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36075 } } }
I20260811 00:31:44.902151 25939 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:44.901695 25938 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2eccc660cec439eb43493ac3c96520b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2eccc660cec439eb43493ac3c96520b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36075 } } }
I20260811 00:31:44.902427 25938 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:44.907444 25945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:31:44.912863 25929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:44.891181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44427 (local address 127.0.0.1:36075)
0811 00:31:44.891380 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:44.891385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:44.891953 (+ 568us) server_negotiation.cc:407] Connection header received
0811 00:31:44.892187 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:44.892193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:44.892284 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:44.892412 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:44.894017 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:44.894902 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:44.896679 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:44.897400 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:44.897492 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:44.901038 (+ 3546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:44.901052 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:44.901059 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:44.901115 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:44.906697 (+ 5582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:44.908050 (+ 1353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:44.908057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:44.908061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:44.908374 (+ 313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:44.909128 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:44.909140 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:44.909144 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:44.909643 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:44.909988 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:44.911872 (+ 1884us) server_negotiation.cc:299] Negotiation successful
0811 00:31:44.912168 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 00:31:44.920655 25945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:31:44.931851 25955 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 00:31:44.937487 25945 catalog_manager.cc:1347] Generated new cluster ID: 7c69e3ab00d44baf9f493f643889e9d0
I20260811 00:31:44.937656 25945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:31:44.959106 25945 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:31:44.960494 25945 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:31:44.972591 25945 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2eccc660cec439eb43493ac3c96520b: Generated new TSK 0
I20260811 00:31:44.973549 25945 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:31:44.987180 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 25847
I20260811 00:31:44.989975 25921 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:45.090955 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 25847
W20260811 00:31:45.114526 25829 connection.cc:535] client connection to 127.0.0.1:36075 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36075): Connection reset by peer (error 104)
2026-08-11T00:31:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:31:45 dist-test-slave-fd7l krb5kdc[25960](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:31:45 dist-test-slave-fd7l krb5kdc[25960](info): set up 2 sockets
Aug 11 00:31:45 dist-test-slave-fd7l krb5kdc[25960](info): commencing operation
krb5kdc: starting...
W20260811 00:31:46.339919 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.006s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:31:46 dist-test-slave-fd7l krb5kdc[25960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:31:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:31:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:31:46.688831 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36173
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:31:47.092939 25977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:47.093295 25977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:47.093397 25977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:47.101413 25977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:31:47.101575 25977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:47.101625 25977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:47.101682 25977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:31:47.101730 25977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:31:47.112696 25977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36173
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:47.114151 25977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:47.115588 25977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:47.129796 25984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:47.130089 25985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:47.130123 25987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:47.131713 25977 server_base.cc:1034] running on GCE node
I20260811 00:31:47.132735 25977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:47.134353 25977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:47.135600 25977 hybrid_clock.cc:648] HybridClock initialized: now 1786408307135553 us; error 34 us; skew 500 ppm
I20260811 00:31:47.135977 25977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:47 dist-test-slave-fd7l krb5kdc[25960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408307, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:47.139670 25977 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:31:47.141785 25977 webserver.cc:460] Webserver started at http://127.0.0.1:39325/ using document root <none> and password file <none>
I20260811 00:31:47.142627 25977 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:47.142755 25977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:47.143051 25977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:47.146574 25977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "18fcb59db9354bf58b97814e523df0f3"
format_stamp: "Formatted at 2026-08-11 00:31:47 on dist-test-slave-fd7l"
server_key: "ced957291c78ae74c9c5df4827d2f442"
server_key_iv: "1153caec00717933722317e4318ae37f"
server_key_version: "encryptionkey@0"
I20260811 00:31:47.147431 25977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "18fcb59db9354bf58b97814e523df0f3"
format_stamp: "Formatted at 2026-08-11 00:31:47 on dist-test-slave-fd7l"
server_key: "ced957291c78ae74c9c5df4827d2f442"
server_key_iv: "1153caec00717933722317e4318ae37f"
server_key_version: "encryptionkey@0"
I20260811 00:31:47.154026 25977 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 00:31:47.158525 25996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:47.160907 25977 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:31:47.161175 25977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "18fcb59db9354bf58b97814e523df0f3"
format_stamp: "Formatted at 2026-08-11 00:31:47 on dist-test-slave-fd7l"
server_key: "ced957291c78ae74c9c5df4827d2f442"
server_key_iv: "1153caec00717933722317e4318ae37f"
server_key_version: "encryptionkey@0"
I20260811 00:31:47.161448 25977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:47.183604 25977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:47.188766 25977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:47.189105 25977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:47.210796 25977 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36173
I20260811 00:31:47.210896 26055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36173 every 8 connection(s)
I20260811 00:31:47.212354 25977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:31:47.216781 26056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:47.219952 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 25977
I20260811 00:31:47.220180 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:31:47.220701 2528 external_mini_cluster.cc:1428] Setting key e4f37d033652845ee3eff5620df8de68
I20260811 00:31:47.230337 26056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3: Bootstrap starting.
I20260811 00:31:47.234583 26056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:47.235918 26056 log.cc:824] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:47.235978 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:47.222811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33583 (local address 127.0.0.1:36173)
0811 00:31:47.223746 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:47.223760 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:47.223803 (+ 43us) server_negotiation.cc:407] Connection header received
0811 00:31:47.225739 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:47.225769 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:47.226281 (+ 512us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:47.226687 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:47.227425 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:47.228901 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:47.230899 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:47.231651 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:47.231805 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:47.233428 (+ 1623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:47.233436 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:47.233453 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:47.234030 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:47.234119 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:47.234148 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:47.234280 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:47.234688 (+ 408us) server_negotiation.cc:299] Negotiation successful
0811 00:31:47.234896 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":426,"threads_started":1}
I20260811 00:31:47.239292 26056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3: No bootstrap required, opened a new log
I20260811 00:31:47.240418 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:31:47.247568 26056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fcb59db9354bf58b97814e523df0f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36173 } }
I20260811 00:31:47.248042 26056 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:47.248202 26056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18fcb59db9354bf58b97814e523df0f3, State: Initialized, Role: FOLLOWER
I20260811 00:31:47.248911 26056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fcb59db9354bf58b97814e523df0f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36173 } }
I20260811 00:31:47.249177 26056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:31:47.249341 26056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:31:47.249538 26056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:47.251994 26056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fcb59db9354bf58b97814e523df0f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36173 } }
I20260811 00:31:47.252223 26059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:47.245101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58863 (local address 127.0.0.1:36173)
0811 00:31:47.245372 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:47.245382 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:47.245938 (+ 556us) server_negotiation.cc:407] Connection header received
0811 00:31:47.246183 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:47.246188 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:47.246257 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:47.246383 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:47.246939 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:47.247730 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:47.249467 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:47.249982 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:47.250136 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:47.251409 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:47.251415 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:47.251418 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:47.251439 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:47.251481 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:47.251498 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:47.251624 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:47.251835 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 00:31:47.251896 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:31:47.252767 26056 leader_election.cc:302] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18fcb59db9354bf58b97814e523df0f3; no voters:
I20260811 00:31:47.253443 26056 leader_election.cc:288] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:31:47.253604 26069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:47.254750 26069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3 [term 1 LEADER]: Becoming Leader. State: Replica: 18fcb59db9354bf58b97814e523df0f3, State: Running, Role: LEADER
I20260811 00:31:47.255303 26069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fcb59db9354bf58b97814e523df0f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36173 } }
I20260811 00:31:47.255923 26056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:31:47.258203 26071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18fcb59db9354bf58b97814e523df0f3. Latest consensus state: current_term: 1 leader_uuid: "18fcb59db9354bf58b97814e523df0f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fcb59db9354bf58b97814e523df0f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36173 } } }
I20260811 00:31:47.258581 26071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3 [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:47.258884 26070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18fcb59db9354bf58b97814e523df0f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fcb59db9354bf58b97814e523df0f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36173 } } }
I20260811 00:31:47.259289 26070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3 [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:47.264701 26077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:31:47.271749 26077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:31:47.285255 26077 catalog_manager.cc:1347] Generated new cluster ID: 835f6e29e39449039aa62457dc093e3d
I20260811 00:31:47.285406 26077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:31:47.321807 26077 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:31:47.323989 26077 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:31:47.336000 26077 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18fcb59db9354bf58b97814e523df0f3: Generated new TSK 0
I20260811 00:31:47.337252 26077 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:31:47.351325 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 25977
W20260811 00:31:47.352936 26051 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33583
W20260811 00:31:47.353818 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 25977: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:31:47.354033 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 25977
W20260811 00:31:47.377904 25958 connection.cc:535] client connection to 127.0.0.1:36173 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36173): Connection reset by peer (error 104)
2026-08-11T00:31:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2263 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:31:47 dist-test-slave-fd7l krb5kdc[26092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:31:47 dist-test-slave-fd7l krb5kdc[26092](info): set up 2 sockets
Aug 11 00:31:47 dist-test-slave-fd7l krb5kdc[26092](info): commencing operation
krb5kdc: starting...
W20260811 00:31:48.628355 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.001s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:31:48 dist-test-slave-fd7l krb5kdc[26092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:31:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:31:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:31:48.989027 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35555
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:31:49.400409 26106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:49.400847 26106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:49.400956 26106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:49.408988 26106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:31:49.409144 26106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:49.409195 26106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:49.409251 26106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:31:49.409299 26106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:31:49.420343 26106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35555
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:49.422099 26106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:49.423658 26106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:49.438213 26111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:49.438187 26113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:49.438585 26115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:49.439469 26106 server_base.cc:1034] running on GCE node
I20260811 00:31:49.440558 26106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:49.442384 26106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:49.443689 26106 hybrid_clock.cc:648] HybridClock initialized: now 1786408309443629 us; error 60 us; skew 500 ppm
I20260811 00:31:49.444152 26106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:49 dist-test-slave-fd7l krb5kdc[26092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408309, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:49.454348 26106 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:31:49.456440 26106 webserver.cc:460] Webserver started at http://127.0.0.1:40383/ using document root <none> and password file <none>
I20260811 00:31:49.457383 26106 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:49.457569 26106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:49.457935 26106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:49.461521 26106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "7b8d993787ee476ca1804466a581bc3e"
format_stamp: "Formatted at 2026-08-11 00:31:49 on dist-test-slave-fd7l"
server_key: "532430c0a89040aacbcbd5df2f7ffee3"
server_key_iv: "10ad436a68a7f02dcb4669da6c8839d8"
server_key_version: "encryptionkey@0"
I20260811 00:31:49.462702 26106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "7b8d993787ee476ca1804466a581bc3e"
format_stamp: "Formatted at 2026-08-11 00:31:49 on dist-test-slave-fd7l"
server_key: "532430c0a89040aacbcbd5df2f7ffee3"
server_key_iv: "10ad436a68a7f02dcb4669da6c8839d8"
server_key_version: "encryptionkey@0"
I20260811 00:31:49.470176 26106 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 00:31:49.475785 26123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:49.477597 26106 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 00:31:49.477845 26106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "7b8d993787ee476ca1804466a581bc3e"
format_stamp: "Formatted at 2026-08-11 00:31:49 on dist-test-slave-fd7l"
server_key: "532430c0a89040aacbcbd5df2f7ffee3"
server_key_iv: "10ad436a68a7f02dcb4669da6c8839d8"
server_key_version: "encryptionkey@0"
I20260811 00:31:49.478109 26106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:49.485594 26106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:49.491221 26106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:49.491611 26106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:49.512401 26106 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35555
I20260811 00:31:49.512454 26184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35555 every 8 connection(s)
I20260811 00:31:49.514000 26106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:31:49.516920 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 26106
I20260811 00:31:49.517132 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:31:49.517597 2528 external_mini_cluster.cc:1428] Setting key 790e1aea82ba6a80e1e1fff50555d4c9
I20260811 00:31:49.518415 26185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:49.528046 26189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:49.519624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50647 (local address 127.0.0.1:35555)
0811 00:31:49.520473 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:49.520490 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:49.520570 (+ 80us) server_negotiation.cc:407] Connection header received
0811 00:31:49.522710 (+ 2140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:49.522744 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:49.523294 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:49.523795 (+ 501us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:49.526107 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:49.526115 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:49.526134 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:49.526152 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:49.526262 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:49.526288 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:49.526543 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:49.526684 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 00:31:49.527006 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":353,"threads_started":1}
I20260811 00:31:49.532399 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:31:49.534011 26185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e: Bootstrap starting.
I20260811 00:31:49.537767 26185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:49.539129 26185 log.cc:824] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:49.539882 26189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:49.537211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43615 (local address 127.0.0.1:35555)
0811 00:31:49.537417 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:49.537424 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:49.537783 (+ 359us) server_negotiation.cc:407] Connection header received
0811 00:31:49.537949 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:49.537955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:49.538022 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:49.538165 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:49.539121 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:49.539127 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:49.539131 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:49.539152 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:49.539203 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:49.539224 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:49.539353 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:49.539468 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 00:31:49.539530 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 00:31:49.542378 26185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e: No bootstrap required, opened a new log
I20260811 00:31:49.550480 26185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b8d993787ee476ca1804466a581bc3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35555 } }
I20260811 00:31:49.550940 26185 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:49.551062 26185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b8d993787ee476ca1804466a581bc3e, State: Initialized, Role: FOLLOWER
I20260811 00:31:49.551707 26185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b8d993787ee476ca1804466a581bc3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35555 } }
I20260811 00:31:49.551950 26185 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:31:49.552119 26185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:31:49.552296 26185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:49.555135 26185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b8d993787ee476ca1804466a581bc3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35555 } }
I20260811 00:31:49.555689 26185 leader_election.cc:302] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b8d993787ee476ca1804466a581bc3e; no voters:
I20260811 00:31:49.556318 26185 leader_election.cc:288] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:31:49.556584 26198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:49.557340 26198 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e [term 1 LEADER]: Becoming Leader. State: Replica: 7b8d993787ee476ca1804466a581bc3e, State: Running, Role: LEADER
I20260811 00:31:49.558127 26198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b8d993787ee476ca1804466a581bc3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35555 } }
I20260811 00:31:49.558892 26185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:31:49.561362 26200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b8d993787ee476ca1804466a581bc3e. Latest consensus state: current_term: 1 leader_uuid: "7b8d993787ee476ca1804466a581bc3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b8d993787ee476ca1804466a581bc3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35555 } } }
I20260811 00:31:49.561914 26200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:49.562333 26199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b8d993787ee476ca1804466a581bc3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b8d993787ee476ca1804466a581bc3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35555 } } }
I20260811 00:31:49.562655 26199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:49.566699 26206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:31:49.573016 26206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:31:49.584303 26206 catalog_manager.cc:1347] Generated new cluster ID: a0ef511be78f4e62a28c5caaf0a9d6ee
I20260811 00:31:49.584478 26206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:31:49.598569 26206 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:31:49.599113 26206 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:31:49.607663 26206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b8d993787ee476ca1804466a581bc3e: Generated new TSK 0
I20260811 00:31:49.608678 26206 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:31:49.626222 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 26106
W20260811 00:31:49.627806 26179 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50647
W20260811 00:31:49.628748 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 26106: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:31:49.628942 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 26106
2026-08-11T00:31:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2278 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:31:49 dist-test-slave-fd7l krb5kdc[26222](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:31:49 dist-test-slave-fd7l krb5kdc[26222](info): set up 2 sockets
Aug 11 00:31:49 dist-test-slave-fd7l krb5kdc[26222](info): commencing operation
krb5kdc: starting...
W20260811 00:31:50.929273 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.014s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:31:51 dist-test-slave-fd7l krb5kdc[26222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:31:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:31:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:31:51.329437 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43881
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:31:51.724937 26237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:51.725327 26237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:51.725430 26237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:51.733371 26237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:31:51.733517 26237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:51.733580 26237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:51.733649 26237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:31:51.733687 26237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:31:51.744437 26237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:51.745924 26237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:51.747450 26237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:51.760818 26242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:51.760833 26245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:51.760859 26243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:51.762598 26237 server_base.cc:1034] running on GCE node
I20260811 00:31:51.764147 26237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:51.766017 26237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:51.767345 26237 hybrid_clock.cc:648] HybridClock initialized: now 1786408311767293 us; error 58 us; skew 500 ppm
I20260811 00:31:51.767874 26237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:51 dist-test-slave-fd7l krb5kdc[26222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408311, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:51.771648 26237 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:31:51.774044 26237 webserver.cc:460] Webserver started at http://127.0.0.1:36985/ using document root <none> and password file <none>
I20260811 00:31:51.774868 26237 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:51.775046 26237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:51.775399 26237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:51.778797 26237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "60f5144aa15f41ccb629086ec127ea43"
format_stamp: "Formatted at 2026-08-11 00:31:51 on dist-test-slave-fd7l"
server_key: "77ea455ed54ad9c6a521b655421a8d51"
server_key_iv: "8c2c475f47152d666dc9e2a869be3085"
server_key_version: "encryptionkey@0"
I20260811 00:31:51.779676 26237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "60f5144aa15f41ccb629086ec127ea43"
format_stamp: "Formatted at 2026-08-11 00:31:51 on dist-test-slave-fd7l"
server_key: "77ea455ed54ad9c6a521b655421a8d51"
server_key_iv: "8c2c475f47152d666dc9e2a869be3085"
server_key_version: "encryptionkey@0"
I20260811 00:31:51.785813 26237 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 00:31:51.790729 26253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:51.793242 26237 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:31:51.793613 26237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "60f5144aa15f41ccb629086ec127ea43"
format_stamp: "Formatted at 2026-08-11 00:31:51 on dist-test-slave-fd7l"
server_key: "77ea455ed54ad9c6a521b655421a8d51"
server_key_iv: "8c2c475f47152d666dc9e2a869be3085"
server_key_version: "encryptionkey@0"
I20260811 00:31:51.793938 26237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:51.815910 26237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:51.821017 26237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:51.821353 26237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:51.843014 26237 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43881
I20260811 00:31:51.843221 26315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43881 every 8 connection(s)
I20260811 00:31:51.844784 26237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:31:51.849710 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 26237
I20260811 00:31:51.849956 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:31:51.850579 2528 external_mini_cluster.cc:1428] Setting key 5dc06f74ff60f3ec8f0b9c7f6830a77b
I20260811 00:31:51.850965 26316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 00:31:51 dist-test-slave-fd7l krb5kdc[26222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 00:31:51.871066 26316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43: Bootstrap starting.
I20260811 00:31:51.875573 26316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:51.876992 26316 log.cc:824] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:51.881047 26316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43: No bootstrap required, opened a new log
I20260811 00:31:51.882625 26319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:51.853289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53769 (local address 127.0.0.1:43881)
0811 00:31:51.854295 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:51.854310 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:51.854374 (+ 64us) server_negotiation.cc:407] Connection header received
0811 00:31:51.857212 (+ 2838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:51.857249 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:51.857848 (+ 599us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:51.858342 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:51.860798 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:51.862436 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:51.865248 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:51.866211 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:51.866411 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:51.873056 (+ 6645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:51.873067 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:51.873085 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:51.873138 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:51.877355 (+ 4217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:51.878389 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:51.878400 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:51.878412 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:51.878774 (+ 362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:51.879355 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:51.879359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:51.879362 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:51.880048 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:51.880306 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:51.880899 (+ 593us) server_negotiation.cc:299] Negotiation successful
0811 00:31:51.881416 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":455,"threads_started":1}
I20260811 00:31:51.887707 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:31:51.893155 26316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f5144aa15f41ccb629086ec127ea43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43881 } }
I20260811 00:31:51.893913 26316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:51.894069 26316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60f5144aa15f41ccb629086ec127ea43, State: Initialized, Role: FOLLOWER
I20260811 00:31:51.894973 26316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f5144aa15f41ccb629086ec127ea43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43881 } }
I20260811 00:31:51.895318 26316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:31:51.895493 26316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:31:51.895713 26316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:51.899187 26316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f5144aa15f41ccb629086ec127ea43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43881 } }
I20260811 00:31:51.899889 26316 leader_election.cc:302] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60f5144aa15f41ccb629086ec127ea43; no voters:
I20260811 00:31:51.900707 26316 leader_election.cc:288] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:31:51.900825 26330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:51.902994 26316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:31:51.907630 26330 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43 [term 1 LEADER]: Becoming Leader. State: Replica: 60f5144aa15f41ccb629086ec127ea43, State: Running, Role: LEADER
I20260811 00:31:51.908330 26330 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f5144aa15f41ccb629086ec127ea43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43881 } }
I20260811 00:31:51.911708 26331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60f5144aa15f41ccb629086ec127ea43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f5144aa15f41ccb629086ec127ea43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43881 } } }
I20260811 00:31:51.912186 26331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43 [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:51.912808 26332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60f5144aa15f41ccb629086ec127ea43. Latest consensus state: current_term: 1 leader_uuid: "60f5144aa15f41ccb629086ec127ea43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60f5144aa15f41ccb629086ec127ea43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43881 } } }
I20260811 00:31:51.913410 26332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43 [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:51.918304 26319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:51.894829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49671 (local address 127.0.0.1:43881)
0811 00:31:51.895104 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:51.895112 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:51.895462 (+ 350us) server_negotiation.cc:407] Connection header received
0811 00:31:51.895743 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:51.895748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:51.895818 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:51.895943 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:51.897926 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:51.898690 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:51.900549 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:51.901317 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:51.901514 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:51.905971 (+ 4457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:51.905981 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:51.905989 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:51.906050 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:51.911199 (+ 5149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:51.912746 (+ 1547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:51.912755 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:51.912761 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:51.912991 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:51.913736 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:51.913744 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:51.913749 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:51.914607 (+ 858us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:51.916902 (+ 2295us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:51.917256 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 00:31:51.917691 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 00:31:51.924238 26341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:31:51.930537 26341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:31:51.941408 26341 catalog_manager.cc:1347] Generated new cluster ID: ceb3b02b3cac47a6aab0a1ce05c442c0
I20260811 00:31:51.941545 26341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:31:51.953904 26341 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:31:51.955667 26341 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:31:51.970005 26341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60f5144aa15f41ccb629086ec127ea43: Generated new TSK 0
I20260811 00:31:51.970968 26341 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:31:51.981137 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 26237
I20260811 00:31:51.982301 26310 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:52.077847 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 26237
2026-08-11T00:31:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2447 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:31:52 dist-test-slave-fd7l krb5kdc[26356](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:31:52 dist-test-slave-fd7l krb5kdc[26356](info): set up 2 sockets
Aug 11 00:31:52 dist-test-slave-fd7l krb5kdc[26356](info): commencing operation
krb5kdc: starting...
W20260811 00:31:53.370146 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:31:53 dist-test-slave-fd7l krb5kdc[26356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:31:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:31:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:31:53.732972 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46539
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:31:54.123148 26372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:54.123498 26372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:54.123611 26372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:54.131184 26372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:31:54.131326 26372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:54.131376 26372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:54.131431 26372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:31:54.131477 26372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:31:54.141876 26372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:54.143313 26372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:54.144706 26372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:54.158573 26378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:54.158874 26377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:54.158821 26380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:54.160691 26372 server_base.cc:1034] running on GCE node
I20260811 00:31:54.161718 26372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:54.163563 26372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:54.164819 26372 hybrid_clock.cc:648] HybridClock initialized: now 1786408314164769 us; error 41 us; skew 500 ppm
I20260811 00:31:54.165225 26372 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:54 dist-test-slave-fd7l krb5kdc[26356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408314, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:54.168963 26372 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:31:54.171060 26372 webserver.cc:460] Webserver started at http://127.0.0.1:40459/ using document root <none> and password file <none>
I20260811 00:31:54.171893 26372 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:54.172013 26372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:54.172303 26372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:54.176868 26372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "1273f989f5b74058bdef3c7ee8c97b38"
format_stamp: "Formatted at 2026-08-11 00:31:54 on dist-test-slave-fd7l"
server_key: "aa8fbc0ed60bb802eb3494b2b12c5051"
server_key_iv: "169aed0cfac92aac67a4c7a3fd9b202a"
server_key_version: "encryptionkey@0"
I20260811 00:31:54.177903 26372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "1273f989f5b74058bdef3c7ee8c97b38"
format_stamp: "Formatted at 2026-08-11 00:31:54 on dist-test-slave-fd7l"
server_key: "aa8fbc0ed60bb802eb3494b2b12c5051"
server_key_iv: "169aed0cfac92aac67a4c7a3fd9b202a"
server_key_version: "encryptionkey@0"
I20260811 00:31:54.184242 26372 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 00:31:54.189281 26388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:54.191068 26372 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:31:54.191275 26372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "1273f989f5b74058bdef3c7ee8c97b38"
format_stamp: "Formatted at 2026-08-11 00:31:54 on dist-test-slave-fd7l"
server_key: "aa8fbc0ed60bb802eb3494b2b12c5051"
server_key_iv: "169aed0cfac92aac67a4c7a3fd9b202a"
server_key_version: "encryptionkey@0"
I20260811 00:31:54.191505 26372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:54.233289 26372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:54.238022 26372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:54.238282 26372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:54.259485 26372 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46539
I20260811 00:31:54.259501 26453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46539 every 8 connection(s)
I20260811 00:31:54.261216 26372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:31:54.265663 26454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:54.270982 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 26372
I20260811 00:31:54.271224 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:31:54.271824 2528 external_mini_cluster.cc:1428] Setting key 80a59624fc219228c11ebe989b067a7b
I20260811 00:31:54.278988 26454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38: Bootstrap starting.
I20260811 00:31:54.282696 26454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:54.283983 26454 log.cc:824] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:54.287242 26454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38: No bootstrap required, opened a new log
I20260811 00:31:54.287263 26457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:54.274114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56085 (local address 127.0.0.1:46539)
0811 00:31:54.275076 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:54.275106 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:54.275156 (+ 50us) server_negotiation.cc:407] Connection header received
0811 00:31:54.277564 (+ 2408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:54.277594 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:54.278092 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:54.278528 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:54.279170 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:54.280682 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:54.282103 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:54.282918 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:54.283165 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:54.284614 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:54.284627 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:54.284652 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:54.285346 (+ 694us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:54.285450 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:54.285484 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:54.285633 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:54.285898 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 00:31:54.286218 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":476,"threads_started":1}
I20260811 00:31:54.291744 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:31:54.296263 26454 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1273f989f5b74058bdef3c7ee8c97b38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46539 } }
I20260811 00:31:54.296914 26454 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:54.297084 26454 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1273f989f5b74058bdef3c7ee8c97b38, State: Initialized, Role: FOLLOWER
I20260811 00:31:54.297997 26454 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1273f989f5b74058bdef3c7ee8c97b38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46539 } }
I20260811 00:31:54.298403 26454 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:31:54.298610 26454 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:31:54.298842 26454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:54.305019 26454 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1273f989f5b74058bdef3c7ee8c97b38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46539 } }
I20260811 00:31:54.305898 26454 leader_election.cc:302] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1273f989f5b74058bdef3c7ee8c97b38; no voters:
I20260811 00:31:54.306795 26454 leader_election.cc:288] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:31:54.306936 26465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:54.308218 26465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38 [term 1 LEADER]: Becoming Leader. State: Replica: 1273f989f5b74058bdef3c7ee8c97b38, State: Running, Role: LEADER
I20260811 00:31:54.308964 26465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1273f989f5b74058bdef3c7ee8c97b38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46539 } }
I20260811 00:31:54.309582 26457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:54.297182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57423 (local address 127.0.0.1:46539)
0811 00:31:54.297501 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:54.297509 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:54.298128 (+ 619us) server_negotiation.cc:407] Connection header received
0811 00:31:54.298426 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:54.298432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:54.298576 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:54.298806 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:54.299677 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:54.301248 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:54.306239 (+ 4991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:54.306901 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:54.307100 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:54.308617 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:54.308624 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:54.308628 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:54.308657 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:54.308716 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:54.308736 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:54.308860 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:54.309022 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 00:31:54.309167 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260811 00:31:54.309587 26454 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:31:54.313566 26466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1273f989f5b74058bdef3c7ee8c97b38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1273f989f5b74058bdef3c7ee8c97b38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46539 } } }
I20260811 00:31:54.313661 26468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1273f989f5b74058bdef3c7ee8c97b38. Latest consensus state: current_term: 1 leader_uuid: "1273f989f5b74058bdef3c7ee8c97b38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1273f989f5b74058bdef3c7ee8c97b38" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46539 } } }
I20260811 00:31:54.314175 26466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38 [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:54.314175 26468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38 [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:54.322273 26477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:31:54.327665 26477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:31:54.337997 26477 catalog_manager.cc:1347] Generated new cluster ID: cb1e36bff3774664bf571e616671de2a
I20260811 00:31:54.338158 26477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:31:54.352890 26477 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:31:54.355049 26477 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:31:54.362648 26477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1273f989f5b74058bdef3c7ee8c97b38: Generated new TSK 0
I20260811 00:31:54.363888 26477 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:31:54.374017 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 26372
W20260811 00:31:54.375407 26449 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56085
W20260811 00:31:54.376252 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 26372: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:31:54.376446 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 26372
2026-08-11T00:31:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:31:54 dist-test-slave-fd7l krb5kdc[26492](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:31:54 dist-test-slave-fd7l krb5kdc[26492](info): set up 2 sockets
Aug 11 00:31:54 dist-test-slave-fd7l krb5kdc[26492](info): commencing operation
krb5kdc: starting...
W20260811 00:31:55.664143 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:31:55 dist-test-slave-fd7l krb5kdc[26492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:31:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:31:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:31:56.028076 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43861
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:31:56.425432 26506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:56.425840 26506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:56.426023 26506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:56.433852 26506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:31:56.434046 26506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:56.434175 26506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:56.434298 26506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:31:56.434378 26506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:31:56.445176 26506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:56.446851 26506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:56.448421 26506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:56.463593 26515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:56.463663 26512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:56.463682 26513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:56.464984 26506 server_base.cc:1034] running on GCE node
I20260811 00:31:56.466012 26506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:56.467844 26506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:56.469164 26506 hybrid_clock.cc:648] HybridClock initialized: now 1786408316469085 us; error 79 us; skew 500 ppm
I20260811 00:31:56.469662 26506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:56 dist-test-slave-fd7l krb5kdc[26492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408316, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:56.473726 26506 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:31:56.475790 26506 webserver.cc:460] Webserver started at http://127.0.0.1:41425/ using document root <none> and password file <none>
I20260811 00:31:56.476841 26506 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:56.477035 26506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:56.477437 26506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:56.481398 26506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "c01a2a247a0442fcbb937f13de381c53"
format_stamp: "Formatted at 2026-08-11 00:31:56 on dist-test-slave-fd7l"
server_key: "d80fc976e33191f0aaa9cf7ba7345a5a"
server_key_iv: "acff528931c3183d801f344d53992b63"
server_key_version: "encryptionkey@0"
I20260811 00:31:56.482445 26506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "c01a2a247a0442fcbb937f13de381c53"
format_stamp: "Formatted at 2026-08-11 00:31:56 on dist-test-slave-fd7l"
server_key: "d80fc976e33191f0aaa9cf7ba7345a5a"
server_key_iv: "acff528931c3183d801f344d53992b63"
server_key_version: "encryptionkey@0"
I20260811 00:31:56.489840 26506 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260811 00:31:56.495672 26522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:56.498052 26506 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 00:31:56.498414 26506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "c01a2a247a0442fcbb937f13de381c53"
format_stamp: "Formatted at 2026-08-11 00:31:56 on dist-test-slave-fd7l"
server_key: "d80fc976e33191f0aaa9cf7ba7345a5a"
server_key_iv: "acff528931c3183d801f344d53992b63"
server_key_version: "encryptionkey@0"
I20260811 00:31:56.498739 26506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:56.506783 26506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:56.512825 26506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:56.513193 26506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:56.535362 26506 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43861
I20260811 00:31:56.535512 26585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43861 every 8 connection(s)
I20260811 00:31:56.537047 26506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:31:56.542397 26586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:56.545984 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 26506
I20260811 00:31:56.546183 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:31:56.546675 2528 external_mini_cluster.cc:1428] Setting key f225e35cc91bbbda8083e5518d1e7070
I20260811 00:31:56.557206 26589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:56.549206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45455 (local address 127.0.0.1:43861)
0811 00:31:56.550223 (+ 1017us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:56.550237 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:56.550287 (+ 50us) server_negotiation.cc:407] Connection header received
0811 00:31:56.552331 (+ 2044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:56.552366 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:56.552951 (+ 585us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:56.553396 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:56.555156 (+ 1760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:56.555165 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:56.555181 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:56.555204 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:56.555295 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:56.555321 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:56.555562 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:56.555698 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 00:31:56.555981 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":760,"thread_start_us":483,"threads_started":1}
I20260811 00:31:56.558504 26586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53: Bootstrap starting.
I20260811 00:31:56.561851 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:31:56.563218 26586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:56.564666 26586 log.cc:824] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53: Log is configured to *not* fsync() on all Append() calls
I20260811 00:31:56.568171 26586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53: No bootstrap required, opened a new log
I20260811 00:31:56.573625 26589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:56.570689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35645 (local address 127.0.0.1:43861)
0811 00:31:56.571009 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:56.571018 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:56.571262 (+ 244us) server_negotiation.cc:407] Connection header received
0811 00:31:56.571439 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:56.571445 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:56.571526 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:56.571673 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:56.572801 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:56.572808 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:56.572813 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:31:56.572841 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:56.572918 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:31:56.572942 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:56.573101 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:56.573206 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 00:31:56.573278 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 00:31:56.578673 26586 raft_consensus.cc:348] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01a2a247a0442fcbb937f13de381c53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43861 } }
I20260811 00:31:56.579222 26586 raft_consensus.cc:374] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:56.579360 26586 raft_consensus.cc:729] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01a2a247a0442fcbb937f13de381c53, State: Initialized, Role: FOLLOWER
I20260811 00:31:56.580140 26586 consensus_queue.cc:260] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01a2a247a0442fcbb937f13de381c53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43861 } }
I20260811 00:31:56.580484 26586 raft_consensus.cc:388] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:31:56.580711 26586 raft_consensus.cc:482] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:31:56.580929 26586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:56.584031 26586 raft_consensus.cc:504] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01a2a247a0442fcbb937f13de381c53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43861 } }
I20260811 00:31:56.584810 26586 leader_election.cc:302] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c01a2a247a0442fcbb937f13de381c53; no voters:
I20260811 00:31:56.585563 26586 leader_election.cc:288] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:31:56.585775 26597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:56.587149 26597 raft_consensus.cc:686] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53 [term 1 LEADER]: Becoming Leader. State: Replica: c01a2a247a0442fcbb937f13de381c53, State: Running, Role: LEADER
I20260811 00:31:56.587841 26597 consensus_queue.cc:237] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01a2a247a0442fcbb937f13de381c53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43861 } }
I20260811 00:31:56.588157 26586 sys_catalog.cc:564] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:31:56.591903 26598 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c01a2a247a0442fcbb937f13de381c53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01a2a247a0442fcbb937f13de381c53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43861 } } }
I20260811 00:31:56.591806 26599 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c01a2a247a0442fcbb937f13de381c53. Latest consensus state: current_term: 1 leader_uuid: "c01a2a247a0442fcbb937f13de381c53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01a2a247a0442fcbb937f13de381c53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43861 } } }
I20260811 00:31:56.592563 26599 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53 [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:56.592567 26598 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53 [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:56.595158 26605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:31:56.602499 26605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:31:56.616698 26605 catalog_manager.cc:1347] Generated new cluster ID: db42b07870a944fe8bbacd17d68d064e
I20260811 00:31:56.616858 26605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:31:56.630594 26605 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:31:56.631182 26605 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:31:56.641028 26605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c01a2a247a0442fcbb937f13de381c53: Generated new TSK 0
I20260811 00:31:56.642086 26605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:31:56.652463 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 26506
W20260811 00:31:56.653931 26579 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45455
W20260811 00:31:56.654815 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 26506: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:31:56.655010 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 26506
2026-08-11T00:31:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2282 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:31:56 dist-test-slave-fd7l krb5kdc[26621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:31:56 dist-test-slave-fd7l krb5kdc[26621](info): set up 2 sockets
Aug 11 00:31:56 dist-test-slave-fd7l krb5kdc[26621](info): commencing operation
krb5kdc: starting...
W20260811 00:31:57.955029 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.002s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:31:58 dist-test-slave-fd7l krb5kdc[26621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:31:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:31:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:31:58.323081 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45965
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:31:58.726356 26637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:31:58.726771 26637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:31:58.726955 26637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:31:58.734906 26637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:31:58.735103 26637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:31:58.735234 26637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:31:58.735350 26637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:31:58.735441 26637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:31:58.749611 26637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:31:58.751372 26637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:31:58.753213 26637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:31:58.768355 26646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:58.768322 26644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:31:58.768625 26643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:31:58.769443 26637 server_base.cc:1034] running on GCE node
I20260811 00:31:58.770490 26637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:31:58.772388 26637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:31:58.773699 26637 hybrid_clock.cc:648] HybridClock initialized: now 1786408318773653 us; error 52 us; skew 500 ppm
I20260811 00:31:58.774180 26637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:31:58 dist-test-slave-fd7l krb5kdc[26621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408318, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:31:58.778419 26637 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:31:58.780727 26637 webserver.cc:460] Webserver started at http://127.0.0.1:36723/ using document root <none> and password file <none>
I20260811 00:31:58.782179 26637 fs_manager.cc:359] Metadata directory not provided
I20260811 00:31:58.782390 26637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:31:58.782752 26637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:31:58.786430 26637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "20cc48498c774c5ab859e309a4af994d"
format_stamp: "Formatted at 2026-08-11 00:31:58 on dist-test-slave-fd7l"
server_key: "91ad86756ae8f1bde8967e11e5d4e751"
server_key_iv: "90758752fee21ed560e954e6d75dcf2a"
server_key_version: "encryptionkey@0"
I20260811 00:31:58.787317 26637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "20cc48498c774c5ab859e309a4af994d"
format_stamp: "Formatted at 2026-08-11 00:31:58 on dist-test-slave-fd7l"
server_key: "91ad86756ae8f1bde8967e11e5d4e751"
server_key_iv: "90758752fee21ed560e954e6d75dcf2a"
server_key_version: "encryptionkey@0"
I20260811 00:31:58.794263 26637 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260811 00:31:58.799201 26655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:31:58.801856 26637 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 00:31:58.802259 26637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "20cc48498c774c5ab859e309a4af994d"
format_stamp: "Formatted at 2026-08-11 00:31:58 on dist-test-slave-fd7l"
server_key: "91ad86756ae8f1bde8967e11e5d4e751"
server_key_iv: "90758752fee21ed560e954e6d75dcf2a"
server_key_version: "encryptionkey@0"
I20260811 00:31:58.802604 26637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:31:58.827939 26637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:31:58.833050 26637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:31:58.833408 26637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:31:58.856009 26637 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45965
I20260811 00:31:58.856086 26717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45965 every 8 connection(s)
I20260811 00:31:58.857657 26637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:31:58.861937 26718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:31:58.866906 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 26637
I20260811 00:31:58.867175 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:31:58.867637 2528 external_mini_cluster.cc:1428] Setting key bb87ac5f40c2db97c2bc543bcffecd7b
I20260811 00:31:58.876165 26718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d: Bootstrap starting.
I20260811 00:31:58.880952 26718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d: Neither blocks nor log segments found. Creating new log.
I20260811 00:31:58.882443 26718 log.cc:824] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:31:58 dist-test-slave-fd7l krb5kdc[26621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 00:31:58.887151 26718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d: No bootstrap required, opened a new log
I20260811 00:31:58.896231 26718 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20cc48498c774c5ab859e309a4af994d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45965 } }
I20260811 00:31:58.896896 26718 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:31:58.897050 26718 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20cc48498c774c5ab859e309a4af994d, State: Initialized, Role: FOLLOWER
I20260811 00:31:58.897914 26718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20cc48498c774c5ab859e309a4af994d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45965 } }
I20260811 00:31:58.898276 26718 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:31:58.898430 26718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:31:58.898669 26718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:31:58.899513 26722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:58.870153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:50463 (local address 127.0.0.1:45965)
0811 00:31:58.871292 (+ 1139us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:58.871307 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:58.871366 (+ 59us) server_negotiation.cc:407] Connection header received
0811 00:31:58.874172 (+ 2806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:58.874212 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:58.874781 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:58.875227 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:58.877547 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:58.879171 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:58.881459 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:58.882431 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:58.882671 (+ 240us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:58.888693 (+ 6022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:58.888706 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:58.888726 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:58.888779 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:58.893370 (+ 4591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:58.894647 (+ 1277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:58.894657 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:58.894665 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:58.894915 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:58.895602 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:58.895624 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:58.895626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:58.896286 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:58.896740 (+ 454us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:58.897404 (+ 664us) server_negotiation.cc:299] Negotiation successful
0811 00:31:58.897838 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":859,"thread_start_us":530,"threads_started":1}
I20260811 00:31:58.902240 26718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20cc48498c774c5ab859e309a4af994d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45965 } }
I20260811 00:31:58.902935 26718 leader_election.cc:302] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20cc48498c774c5ab859e309a4af994d; no voters:
I20260811 00:31:58.903754 26718 leader_election.cc:288] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:31:58.903939 26725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:31:58.905540 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:31:58.906512 26718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:31:58.911311 26725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d [term 1 LEADER]: Becoming Leader. State: Replica: 20cc48498c774c5ab859e309a4af994d, State: Running, Role: LEADER
I20260811 00:31:58.912436 26725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20cc48498c774c5ab859e309a4af994d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45965 } }
I20260811 00:31:58.922446 26727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20cc48498c774c5ab859e309a4af994d. Latest consensus state: current_term: 1 leader_uuid: "20cc48498c774c5ab859e309a4af994d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20cc48498c774c5ab859e309a4af994d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45965 } } }
I20260811 00:31:58.922425 26726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20cc48498c774c5ab859e309a4af994d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20cc48498c774c5ab859e309a4af994d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45965 } } }
I20260811 00:31:58.923507 26726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d [sys.catalog]: This master's current role is: LEADER
I20260811 00:31:58.923507 26727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d [sys.catalog]: This master's current role is: LEADER
W20260811 00:31:58.935092 26749 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:31:58.935359 26749 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:31:58.935709 26747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:31:58.938620 26747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:31:58.938969 26722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:31:58.916319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:36683 (local address 127.0.0.1:45965)
0811 00:31:58.916588 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 00:31:58.916598 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:31:58.916622 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:31:58.916700 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:31:58.916704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:31:58.916780 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:31:58.916939 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:31:58.919335 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:58.920339 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:58.922721 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:31:58.923574 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:31:58.923717 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:31:58.927687 (+ 3970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:31:58.927697 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:31:58.927703 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:31:58.927760 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:31:58.932796 (+ 5036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:58.934636 (+ 1840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:58.934647 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:58.934654 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:58.935010 (+ 356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:31:58.936178 (+ 1168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:31:58.936183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:31:58.936186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:31:58.936848 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:31:58.937439 (+ 591us) server_negotiation.cc:1035] Waiting for connection context
0811 00:31:58.938151 (+ 712us) server_negotiation.cc:299] Negotiation successful
0811 00:31:58.938444 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:31:58.952013 26747 catalog_manager.cc:1347] Generated new cluster ID: ea8ef0ba341845df90fa392e458b0513
I20260811 00:31:58.952191 26747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:31:59.003094 26747 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:31:59.005291 26747 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:31:59.017717 26747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20cc48498c774c5ab859e309a4af994d: Generated new TSK 0
I20260811 00:31:59.018844 26747 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:31:59.034128 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 26637
I20260811 00:31:59.035686 26711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:31:59.140666 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 26637
W20260811 00:31:59.166357 26619 connection.cc:535] client connection to 127.0.0.1:45965 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45965): Connection reset by peer (error 104)
2026-08-11T00:31:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2485 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:31:59 dist-test-slave-fd7l krb5kdc[26755](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:31:59 dist-test-slave-fd7l krb5kdc[26755](info): set up 2 sockets
Aug 11 00:31:59 dist-test-slave-fd7l krb5kdc[26755](info): commencing operation
krb5kdc: starting...
W20260811 00:32:00.456291 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:32:00 dist-test-slave-fd7l krb5kdc[26755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:32:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:32:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:32:00.835991 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33349
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:32:01.251952 26768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:32:01.252300 26768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:32:01.252471 26768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:32:01.260283 26768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:32:01.260428 26768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:32:01.260512 26768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:32:01.260675 26768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:32:01.260737 26768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:32:01.271471 26768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:32:01.273185 26768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:32:01.274763 26768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:32:01.288940 26775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:01.289037 26776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:01.289012 26778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:32:01.290089 26768 server_base.cc:1034] running on GCE node
I20260811 00:32:01.291023 26768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:32:01.292843 26768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:32:01.294176 26768 hybrid_clock.cc:648] HybridClock initialized: now 1786408321294150 us; error 70 us; skew 500 ppm
I20260811 00:32:01.294670 26768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:32:01 dist-test-slave-fd7l krb5kdc[26755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408321, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:32:01.304435 26768 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:32:01.306509 26768 webserver.cc:460] Webserver started at http://127.0.0.1:45361/ using document root <none> and password file <none>
I20260811 00:32:01.307410 26768 fs_manager.cc:359] Metadata directory not provided
I20260811 00:32:01.307597 26768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:32:01.307977 26768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:32:01.311612 26768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "ff19e5c9d4c84c1583ec7281cc6169d1"
format_stamp: "Formatted at 2026-08-11 00:32:01 on dist-test-slave-fd7l"
server_key: "f48b94147c4f48ba81669769721e0c91"
server_key_iv: "dc5acd3d39544f194b6c7bed47fc91f9"
server_key_version: "encryptionkey@0"
I20260811 00:32:01.312909 26768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "ff19e5c9d4c84c1583ec7281cc6169d1"
format_stamp: "Formatted at 2026-08-11 00:32:01 on dist-test-slave-fd7l"
server_key: "f48b94147c4f48ba81669769721e0c91"
server_key_iv: "dc5acd3d39544f194b6c7bed47fc91f9"
server_key_version: "encryptionkey@0"
I20260811 00:32:01.320614 26768 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260811 00:32:01.325546 26785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:32:01.327744 26768 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260811 00:32:01.328042 26768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "ff19e5c9d4c84c1583ec7281cc6169d1"
format_stamp: "Formatted at 2026-08-11 00:32:01 on dist-test-slave-fd7l"
server_key: "f48b94147c4f48ba81669769721e0c91"
server_key_iv: "dc5acd3d39544f194b6c7bed47fc91f9"
server_key_version: "encryptionkey@0"
I20260811 00:32:01.328328 26768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:32:01.343991 26768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:32:01.349854 26768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:32:01.350221 26768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:32:01.374529 26768 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33349
I20260811 00:32:01.374570 26844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33349 every 8 connection(s)
I20260811 00:32:01.376349 26768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:32:01.381064 26845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:32:01.386349 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 26768
I20260811 00:32:01.386605 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:32:01.387125 2528 external_mini_cluster.cc:1428] Setting key dea1be3e56656290ab4cbd43583426bb
I20260811 00:32:01.396696 26845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1: Bootstrap starting.
I20260811 00:32:01.401777 26845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1: Neither blocks nor log segments found. Creating new log.
I20260811 00:32:01.403308 26845 log.cc:824] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1: Log is configured to *not* fsync() on all Append() calls
I20260811 00:32:01.407354 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:01.389552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:40993 (local address 127.0.0.1:33349)
0811 00:32:01.390524 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:01.390544 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:01.390608 (+ 64us) server_negotiation.cc:407] Connection header received
0811 00:32:01.394266 (+ 3658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:01.394315 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:01.395048 (+ 733us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:01.395657 (+ 609us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:01.396357 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:01.398692 (+ 2335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:01.400732 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:01.402056 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:01.402363 (+ 307us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:01.403993 (+ 1630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:01.404014 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:01.404049 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:32:01.404896 (+ 847us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:01.404991 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:32:01.405092 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:01.405308 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:01.405652 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 00:32:01.405870 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":376,"threads_started":1}
I20260811 00:32:01.407691 26845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1: No bootstrap required, opened a new log
I20260811 00:32:01.412881 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:32:01.417371 26845 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff19e5c9d4c84c1583ec7281cc6169d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33349 } }
I20260811 00:32:01.417908 26845 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:32:01.418035 26845 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff19e5c9d4c84c1583ec7281cc6169d1, State: Initialized, Role: FOLLOWER
I20260811 00:32:01.418864 26845 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff19e5c9d4c84c1583ec7281cc6169d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33349 } }
I20260811 00:32:01.419188 26845 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:32:01.419387 26845 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:32:01.419643 26845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:32:01.423307 26845 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff19e5c9d4c84c1583ec7281cc6169d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33349 } }
I20260811 00:32:01.424144 26845 leader_election.cc:302] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff19e5c9d4c84c1583ec7281cc6169d1; no voters:
I20260811 00:32:01.425067 26845 leader_election.cc:288] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:32:01.425256 26858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:32:01.426153 26858 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1 [term 1 LEADER]: Becoming Leader. State: Replica: ff19e5c9d4c84c1583ec7281cc6169d1, State: Running, Role: LEADER
I20260811 00:32:01.426890 26858 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff19e5c9d4c84c1583ec7281cc6169d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33349 } }
I20260811 00:32:01.427870 26845 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:32:01.428260 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:01.418506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:59903 (local address 127.0.0.1:33349)
0811 00:32:01.418758 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:01.418765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:01.419825 (+ 1060us) server_negotiation.cc:407] Connection header received
0811 00:32:01.420085 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:01.420089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:01.420175 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:01.420285 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:01.421028 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:01.421925 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:01.424100 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:01.424886 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:01.425096 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:01.426841 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:01.426856 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:01.426862 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:32:01.426910 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:01.426992 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:32:01.427023 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:01.427164 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:01.427777 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 00:32:01.427863 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 00:32:01.430401 26860 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff19e5c9d4c84c1583ec7281cc6169d1. Latest consensus state: current_term: 1 leader_uuid: "ff19e5c9d4c84c1583ec7281cc6169d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff19e5c9d4c84c1583ec7281cc6169d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33349 } } }
I20260811 00:32:01.430940 26860 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:01.431645 26859 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff19e5c9d4c84c1583ec7281cc6169d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff19e5c9d4c84c1583ec7281cc6169d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33349 } } }
I20260811 00:32:01.432226 26859 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:01.433928 26866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:32:01.440909 26866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:32:01.451877 26866 catalog_manager.cc:1347] Generated new cluster ID: 0a8a247e7f9343d39024bbdd3dae3fec
I20260811 00:32:01.452060 26866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:32:01.471287 26866 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:32:01.473448 26866 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:32:01.492828 26866 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff19e5c9d4c84c1583ec7281cc6169d1: Generated new TSK 0
I20260811 00:32:01.494132 26866 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:32:01.515029 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 26768
W20260811 00:32:01.516614 26840 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.241:40993
W20260811 00:32:01.517608 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 26768: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:32:01.517797 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 26768
W20260811 00:32:01.542671 26753 connection.cc:535] client connection to 127.0.0.1:33349 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33349): Connection reset by peer (error 104)
2026-08-11T00:32:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2375 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:32:01 dist-test-slave-fd7l krb5kdc[26884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:32:01 dist-test-slave-fd7l krb5kdc[26884](info): set up 2 sockets
Aug 11 00:32:01 dist-test-slave-fd7l krb5kdc[26884](info): commencing operation
krb5kdc: starting...
W20260811 00:32:02.785255 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.001s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:32:03 dist-test-slave-fd7l krb5kdc[26884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:32:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:32:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:32:03.182446 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46835
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:32:03.592311 26900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:32:03.592845 26900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:32:03.593034 26900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:32:03.601502 26900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:32:03.601644 26900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:32:03.601729 26900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:32:03.601846 26900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:32:03.601933 26900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:32:03.612932 26900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:32:03.614420 26900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:32:03.615875 26900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:32:03.629458 26906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:03.629369 26907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:03.629446 26909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:32:03.630460 26900 server_base.cc:1034] running on GCE node
I20260811 00:32:03.631589 26900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:32:03.633678 26900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:32:03.635025 26900 hybrid_clock.cc:648] HybridClock initialized: now 1786408323634972 us; error 44 us; skew 500 ppm
I20260811 00:32:03.635526 26900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:32:03 dist-test-slave-fd7l krb5kdc[26884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408323, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:32:03.639632 26900 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:32:03.641762 26900 webserver.cc:460] Webserver started at http://127.0.0.1:42041/ using document root <none> and password file <none>
I20260811 00:32:03.642834 26900 fs_manager.cc:359] Metadata directory not provided
I20260811 00:32:03.643046 26900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:32:03.643573 26900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:32:03.647490 26900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "b7bd643d31af428982834882ff1ff6b9"
format_stamp: "Formatted at 2026-08-11 00:32:03 on dist-test-slave-fd7l"
server_key: "d758e01da0b2d1bc4e2758ef5ee5c088"
server_key_iv: "6c474e21c73748f880f9d04b5ab49179"
server_key_version: "encryptionkey@0"
I20260811 00:32:03.648476 26900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "b7bd643d31af428982834882ff1ff6b9"
format_stamp: "Formatted at 2026-08-11 00:32:03 on dist-test-slave-fd7l"
server_key: "d758e01da0b2d1bc4e2758ef5ee5c088"
server_key_iv: "6c474e21c73748f880f9d04b5ab49179"
server_key_version: "encryptionkey@0"
I20260811 00:32:03.654960 26900 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260811 00:32:03.659170 26918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:32:03.661056 26900 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 00:32:03.661309 26900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "b7bd643d31af428982834882ff1ff6b9"
format_stamp: "Formatted at 2026-08-11 00:32:03 on dist-test-slave-fd7l"
server_key: "d758e01da0b2d1bc4e2758ef5ee5c088"
server_key_iv: "6c474e21c73748f880f9d04b5ab49179"
server_key_version: "encryptionkey@0"
I20260811 00:32:03.661604 26900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:32:03.668875 26900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:32:03.674448 26900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:32:03.674842 26900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:32:03.696007 26900 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46835
I20260811 00:32:03.696035 26978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46835 every 8 connection(s)
I20260811 00:32:03.697782 26900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:32:03.702029 26979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:32:03.703466 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 26900
I20260811 00:32:03.703680 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:32:03.704325 2528 external_mini_cluster.cc:1428] Setting key fd72ca378a98fb96640d72c574cfeaa2
I20260811 00:32:03.713511 26982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:03.706628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:38573 (local address 127.0.0.1:46835)
0811 00:32:03.707552 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:03.707565 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:03.707613 (+ 48us) server_negotiation.cc:407] Connection header received
0811 00:32:03.709432 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:03.709462 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:03.709904 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:03.710306 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:03.711742 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:03.711749 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:03.711764 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:32:03.711789 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:03.711877 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:32:03.711904 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:03.712100 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:03.712206 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 00:32:03.712445 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":394,"threads_started":1}
I20260811 00:32:03.717809 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:32:03.718274 26979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9: Bootstrap starting.
I20260811 00:32:03.721817 26979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9: Neither blocks nor log segments found. Creating new log.
I20260811 00:32:03.723444 26979 log.cc:824] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9: Log is configured to *not* fsync() on all Append() calls
I20260811 00:32:03.725657 26982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:03.722625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:45559 (local address 127.0.0.1:46835)
0811 00:32:03.722859 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:03.722866 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:03.723179 (+ 313us) server_negotiation.cc:407] Connection header received
0811 00:32:03.723412 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:03.723416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:03.723477 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:03.723560 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:03.724753 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:03.724759 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:03.724764 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:32:03.724786 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:03.724851 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:32:03.724880 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:03.725045 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:03.725228 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 00:32:03.725314 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 00:32:03.727742 26979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9: No bootstrap required, opened a new log
I20260811 00:32:03.736479 26979 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7bd643d31af428982834882ff1ff6b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46835 } }
I20260811 00:32:03.737103 26979 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:32:03.737246 26979 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7bd643d31af428982834882ff1ff6b9, State: Initialized, Role: FOLLOWER
I20260811 00:32:03.738026 26979 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7bd643d31af428982834882ff1ff6b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46835 } }
I20260811 00:32:03.738286 26979 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:32:03.738435 26979 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:32:03.738646 26979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:32:03.741559 26979 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7bd643d31af428982834882ff1ff6b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46835 } }
I20260811 00:32:03.742220 26979 leader_election.cc:302] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7bd643d31af428982834882ff1ff6b9; no voters:
I20260811 00:32:03.742909 26979 leader_election.cc:288] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:32:03.743131 26989 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:32:03.744053 26989 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9 [term 1 LEADER]: Becoming Leader. State: Replica: b7bd643d31af428982834882ff1ff6b9, State: Running, Role: LEADER
I20260811 00:32:03.744951 26989 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7bd643d31af428982834882ff1ff6b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46835 } }
I20260811 00:32:03.745368 26979 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:32:03.748309 26990 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7bd643d31af428982834882ff1ff6b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7bd643d31af428982834882ff1ff6b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46835 } } }
I20260811 00:32:03.749082 26990 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:03.748487 26991 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7bd643d31af428982834882ff1ff6b9. Latest consensus state: current_term: 1 leader_uuid: "b7bd643d31af428982834882ff1ff6b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7bd643d31af428982834882ff1ff6b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46835 } } }
I20260811 00:32:03.749437 26991 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:03.750775 26998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:32:03.758394 26998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:32:03.769726 26998 catalog_manager.cc:1347] Generated new cluster ID: 128d2def1b2d42f1a7ec06ee0ccc57c7
I20260811 00:32:03.769868 26998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:32:03.784757 26998 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:32:03.785326 26998 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:32:03.800077 26998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7bd643d31af428982834882ff1ff6b9: Generated new TSK 0
I20260811 00:32:03.801849 26998 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:32:03.823226 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 26900
W20260811 00:32:03.824908 26973 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.241:38573
W20260811 00:32:03.825688 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 26900: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:32:03.825862 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 26900
2026-08-11T00:32:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2307 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:32:03 dist-test-slave-fd7l krb5kdc[27011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:32:03 dist-test-slave-fd7l krb5kdc[27011](info): set up 2 sockets
Aug 11 00:32:03 dist-test-slave-fd7l krb5kdc[27011](info): commencing operation
krb5kdc: starting...
W20260811 00:32:05.111088 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.000s sys 0.059s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:32:05 dist-test-slave-fd7l krb5kdc[27011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:32:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:32:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:32:05.465670 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45697
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:32:05.853857 27026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:32:05.854225 27026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:32:05.854418 27026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:32:05.862447 27026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:32:05.862608 27026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:32:05.862720 27026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:32:05.862819 27026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:32:05.862865 27026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:32:05.873864 27026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45697
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:32:05.875455 27026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:32:05.877018 27026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:32:05.891778 27037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:05.891721 27034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:05.891739 27035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:32:05.892839 27026 server_base.cc:1034] running on GCE node
I20260811 00:32:05.894019 27026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:32:05.895843 27026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:32:05.897149 27026 hybrid_clock.cc:648] HybridClock initialized: now 1786408325897082 us; error 70 us; skew 500 ppm
I20260811 00:32:05.897590 27026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:32:05 dist-test-slave-fd7l krb5kdc[27011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408325, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:32:05.901290 27026 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:32:05.903208 27026 webserver.cc:460] Webserver started at http://127.0.0.1:38477/ using document root <none> and password file <none>
I20260811 00:32:05.904024 27026 fs_manager.cc:359] Metadata directory not provided
I20260811 00:32:05.904194 27026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:32:05.904657 27026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:32:05.907964 27026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "d76c7c3256ae4fef9ae84b2a8e20f2f8"
format_stamp: "Formatted at 2026-08-11 00:32:05 on dist-test-slave-fd7l"
server_key: "f5e1843fc06c0ebe78e1918d481bc772"
server_key_iv: "a8f083607b218459eb7870cf3358c172"
server_key_version: "encryptionkey@0"
I20260811 00:32:05.908950 27026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "d76c7c3256ae4fef9ae84b2a8e20f2f8"
format_stamp: "Formatted at 2026-08-11 00:32:05 on dist-test-slave-fd7l"
server_key: "f5e1843fc06c0ebe78e1918d481bc772"
server_key_iv: "a8f083607b218459eb7870cf3358c172"
server_key_version: "encryptionkey@0"
I20260811 00:32:05.915308 27026 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.005s
I20260811 00:32:05.919217 27045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:32:05.920746 27026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 00:32:05.920938 27026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "d76c7c3256ae4fef9ae84b2a8e20f2f8"
format_stamp: "Formatted at 2026-08-11 00:32:05 on dist-test-slave-fd7l"
server_key: "f5e1843fc06c0ebe78e1918d481bc772"
server_key_iv: "a8f083607b218459eb7870cf3358c172"
server_key_version: "encryptionkey@0"
I20260811 00:32:05.921159 27026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:32:05.939172 27026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:32:05.943960 27026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:32:05.944386 27026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:32:05.966140 27026 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45697
I20260811 00:32:05.966182 27099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45697 every 8 connection(s)
I20260811 00:32:05.967815 27026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:32:05.973632 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 27026
I20260811 00:32:05.973778 27100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:32:05.973874 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:32:05.974429 2528 external_mini_cluster.cc:1428] Setting key dfcbae15ea46249452cbbba76231ed58
I20260811 00:32:05.992801 27100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8: Bootstrap starting.
Aug 11 00:32:05 dist-test-slave-fd7l krb5kdc[27011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 00:32:05.996991 27100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8: Neither blocks nor log segments found. Creating new log.
I20260811 00:32:05.998239 27100 log.cc:824] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8: Log is configured to *not* fsync() on all Append() calls
I20260811 00:32:06.002076 27100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8: No bootstrap required, opened a new log
I20260811 00:32:06.004184 27104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:05.976684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:35081 (local address 127.0.0.1:45697)
0811 00:32:05.977660 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:05.977676 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:05.977725 (+ 49us) server_negotiation.cc:407] Connection header received
0811 00:32:05.979461 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:05.979492 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:05.980049 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:05.980478 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:05.983102 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:05.985064 (+ 1962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:05.987323 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:05.988410 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:05.988757 (+ 347us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:05.994502 (+ 5745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:05.994512 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:05.994530 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:05.994583 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:05.998489 (+ 3906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:05.999579 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:05.999587 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:05.999594 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:05.999776 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:06.000552 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:06.000562 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:06.000567 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:06.001484 (+ 917us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:06.001909 (+ 425us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:06.002564 (+ 655us) server_negotiation.cc:299] Negotiation successful
0811 00:32:06.002991 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":440,"threads_started":1}
I20260811 00:32:06.009855 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:32:06.012193 27100 raft_consensus.cc:348] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d76c7c3256ae4fef9ae84b2a8e20f2f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45697 } }
I20260811 00:32:06.012740 27100 raft_consensus.cc:374] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:32:06.012863 27100 raft_consensus.cc:729] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d76c7c3256ae4fef9ae84b2a8e20f2f8, State: Initialized, Role: FOLLOWER
I20260811 00:32:06.013741 27100 consensus_queue.cc:260] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d76c7c3256ae4fef9ae84b2a8e20f2f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45697 } }
I20260811 00:32:06.014026 27100 raft_consensus.cc:388] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:32:06.014182 27100 raft_consensus.cc:482] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:32:06.014365 27100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:32:06.017390 27100 raft_consensus.cc:504] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d76c7c3256ae4fef9ae84b2a8e20f2f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45697 } }
I20260811 00:32:06.018000 27100 leader_election.cc:302] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d76c7c3256ae4fef9ae84b2a8e20f2f8; no voters:
I20260811 00:32:06.019145 27100 leader_election.cc:288] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:32:06.019253 27111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:32:06.022390 27100 sys_catalog.cc:564] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:32:06.025992 27111 raft_consensus.cc:686] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8 [term 1 LEADER]: Becoming Leader. State: Replica: d76c7c3256ae4fef9ae84b2a8e20f2f8, State: Running, Role: LEADER
I20260811 00:32:06.026935 27111 consensus_queue.cc:237] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d76c7c3256ae4fef9ae84b2a8e20f2f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45697 } }
I20260811 00:32:06.031917 27104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:06.015279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:56471 (local address 127.0.0.1:45697)
0811 00:32:06.015504 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:06.015513 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:06.015921 (+ 408us) server_negotiation.cc:407] Connection header received
0811 00:32:06.016153 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:06.016157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:06.016236 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:06.016362 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:06.017924 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:06.018788 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:06.020705 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:06.021200 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:06.021281 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:06.024263 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:06.024276 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:06.024283 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:06.024340 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:06.028105 (+ 3765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:06.029319 (+ 1214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:06.029324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:06.029327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:06.029478 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:06.030006 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:06.030010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:06.030012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:06.030357 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:06.030708 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:06.031506 (+ 798us) server_negotiation.cc:299] Negotiation successful
0811 00:32:06.031651 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 00:32:06.037703 27113 sys_catalog.cc:455] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d76c7c3256ae4fef9ae84b2a8e20f2f8. Latest consensus state: current_term: 1 leader_uuid: "d76c7c3256ae4fef9ae84b2a8e20f2f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d76c7c3256ae4fef9ae84b2a8e20f2f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45697 } } }
I20260811 00:32:06.038223 27113 sys_catalog.cc:458] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:06.038766 27114 sys_catalog.cc:455] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d76c7c3256ae4fef9ae84b2a8e20f2f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d76c7c3256ae4fef9ae84b2a8e20f2f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45697 } } }
I20260811 00:32:06.040349 27114 sys_catalog.cc:458] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:06.042423 27128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:32:06.047536 27128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:32:06.058583 27128 catalog_manager.cc:1347] Generated new cluster ID: 10a1d1a1c642408a8dc57688cd6fd932
I20260811 00:32:06.058763 27128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:32:06.076804 27128 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:32:06.078208 27128 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:32:06.085825 27128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d76c7c3256ae4fef9ae84b2a8e20f2f8: Generated new TSK 0
I20260811 00:32:06.086923 27128 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:32:06.103036 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 27026
I20260811 00:32:06.105028 27094 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:32:06.202987 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 27026
W20260811 00:32:06.226915 27009 connection.cc:535] client connection to 127.0.0.1:45697 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45697): Connection reset by peer (error 104)
2026-08-11T00:32:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2382 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:32:06 dist-test-slave-fd7l krb5kdc[27138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:32:06 dist-test-slave-fd7l krb5kdc[27138](info): set up 2 sockets
Aug 11 00:32:06 dist-test-slave-fd7l krb5kdc[27138](info): commencing operation
krb5kdc: starting...
W20260811 00:32:07.460114 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:32:07 dist-test-slave-fd7l krb5kdc[27138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:32:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:32:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:32:07.816869 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37137
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:32:08.207878 27152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:32:08.208227 27152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:32:08.208328 27152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:32:08.216045 27152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:32:08.216194 27152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:32:08.216243 27152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:32:08.216296 27152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:32:08.216342 27152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:32:08.226953 27152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37137
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:32:08.228368 27152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:32:08.229846 27152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:32:08.242539 27160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:08.242520 27159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:08.242743 27162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:32:08.243321 27152 server_base.cc:1034] running on GCE node
I20260811 00:32:08.244259 27152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:32:08.245956 27152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:32:08.247201 27152 hybrid_clock.cc:648] HybridClock initialized: now 1786408328247140 us; error 54 us; skew 500 ppm
I20260811 00:32:08.247586 27152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:32:08 dist-test-slave-fd7l krb5kdc[27138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408328, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:32:08.250967 27152 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:32:08.252789 27152 webserver.cc:460] Webserver started at http://127.0.0.1:40131/ using document root <none> and password file <none>
I20260811 00:32:08.253561 27152 fs_manager.cc:359] Metadata directory not provided
I20260811 00:32:08.253682 27152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:32:08.253963 27152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:32:08.257453 27152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "cb415ee2f4714c15825fb5fc7105d898"
format_stamp: "Formatted at 2026-08-11 00:32:08 on dist-test-slave-fd7l"
server_key: "a7bd272861a0e91f8a15f683bd104db0"
server_key_iv: "980fab8fcf8f26f246bcbc98fe60419b"
server_key_version: "encryptionkey@0"
I20260811 00:32:08.258311 27152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "cb415ee2f4714c15825fb5fc7105d898"
format_stamp: "Formatted at 2026-08-11 00:32:08 on dist-test-slave-fd7l"
server_key: "a7bd272861a0e91f8a15f683bd104db0"
server_key_iv: "980fab8fcf8f26f246bcbc98fe60419b"
server_key_version: "encryptionkey@0"
I20260811 00:32:08.264469 27152 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 00:32:08.268445 27170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:32:08.269979 27152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:32:08.270200 27152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "cb415ee2f4714c15825fb5fc7105d898"
format_stamp: "Formatted at 2026-08-11 00:32:08 on dist-test-slave-fd7l"
server_key: "a7bd272861a0e91f8a15f683bd104db0"
server_key_iv: "980fab8fcf8f26f246bcbc98fe60419b"
server_key_version: "encryptionkey@0"
I20260811 00:32:08.270426 27152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:32:08.303699 27152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:32:08.309556 27152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:32:08.309909 27152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:32:08.331197 27152 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37137
I20260811 00:32:08.331228 27231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37137 every 8 connection(s)
I20260811 00:32:08.332880 27152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:32:08.333630 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 27152
I20260811 00:32:08.333885 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:32:08.334601 2528 external_mini_cluster.cc:1428] Setting key 8d970d024b8ac335a03fdca9973a679a
I20260811 00:32:08.338245 27232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:32:08.349784 27235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:08.336891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:41901 (local address 127.0.0.1:37137)
0811 00:32:08.337768 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:08.337780 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:08.337825 (+ 45us) server_negotiation.cc:407] Connection header received
0811 00:32:08.339684 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:08.339712 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:08.340156 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:08.340640 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:08.341277 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:08.342705 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:08.344685 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:08.345512 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:08.345737 (+ 225us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:08.347063 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:08.347072 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:08.347089 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:32:08.347739 (+ 650us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:08.347838 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:32:08.347869 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:08.348089 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:08.348408 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 00:32:08.348686 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":372,"threads_started":1}
I20260811 00:32:08.350911 27232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898: Bootstrap starting.
I20260811 00:32:08.355662 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:32:08.355898 27232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898: Neither blocks nor log segments found. Creating new log.
I20260811 00:32:08.357445 27232 log.cc:824] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898: Log is configured to *not* fsync() on all Append() calls
I20260811 00:32:08.361481 27232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898: No bootstrap required, opened a new log
I20260811 00:32:08.368597 27235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:08.361418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:51479 (local address 127.0.0.1:37137)
0811 00:32:08.361638 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:08.361644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:08.361986 (+ 342us) server_negotiation.cc:407] Connection header received
0811 00:32:08.362150 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:08.362157 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:08.362252 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:08.362388 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:08.362975 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:08.364186 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:08.365647 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:08.366361 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:08.366493 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:08.367662 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:08.367669 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:08.367674 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:32:08.367703 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:08.367766 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:32:08.367786 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:08.367916 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:08.368144 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 00:32:08.368199 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:32:08.371812 27232 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb415ee2f4714c15825fb5fc7105d898" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37137 } }
I20260811 00:32:08.372244 27232 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:32:08.372313 27232 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb415ee2f4714c15825fb5fc7105d898, State: Initialized, Role: FOLLOWER
I20260811 00:32:08.373054 27232 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb415ee2f4714c15825fb5fc7105d898" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37137 } }
I20260811 00:32:08.373356 27232 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:32:08.373546 27232 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:32:08.373744 27232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:32:08.376367 27232 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb415ee2f4714c15825fb5fc7105d898" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37137 } }
I20260811 00:32:08.376991 27232 leader_election.cc:302] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb415ee2f4714c15825fb5fc7105d898; no voters:
I20260811 00:32:08.377609 27232 leader_election.cc:288] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:32:08.377801 27242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:32:08.378504 27242 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898 [term 1 LEADER]: Becoming Leader. State: Replica: cb415ee2f4714c15825fb5fc7105d898, State: Running, Role: LEADER
I20260811 00:32:08.379057 27242 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb415ee2f4714c15825fb5fc7105d898" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37137 } }
I20260811 00:32:08.379873 27232 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:32:08.382292 27244 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb415ee2f4714c15825fb5fc7105d898. Latest consensus state: current_term: 1 leader_uuid: "cb415ee2f4714c15825fb5fc7105d898" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb415ee2f4714c15825fb5fc7105d898" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37137 } } }
I20260811 00:32:08.382784 27244 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:08.383083 27243 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb415ee2f4714c15825fb5fc7105d898" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb415ee2f4714c15825fb5fc7105d898" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37137 } } }
I20260811 00:32:08.383534 27243 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:08.385084 27249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:32:08.390597 27249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:32:08.402048 27249 catalog_manager.cc:1347] Generated new cluster ID: dbf917186d0b469983af3d4d50e8fb39
I20260811 00:32:08.402225 27249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:32:08.413899 27249 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:32:08.415968 27249 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:32:08.431734 27249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb415ee2f4714c15825fb5fc7105d898: Generated new TSK 0
I20260811 00:32:08.433017 27249 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:32:08.442865 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 27152
W20260811 00:32:08.446381 27227 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.241:41901
W20260811 00:32:08.447958 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 27152: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:32:08.448237 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 27152
W20260811 00:32:08.474054 27136 connection.cc:535] client connection to 127.0.0.1:37137 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37137): Connection reset by peer (error 104)
2026-08-11T00:32:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2241 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:32:08 dist-test-slave-fd7l krb5kdc[27266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:32:08 dist-test-slave-fd7l krb5kdc[27266](info): set up 2 sockets
Aug 11 00:32:08 dist-test-slave-fd7l krb5kdc[27266](info): commencing operation
krb5kdc: starting...
W20260811 00:32:09.731211 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:32:09 dist-test-slave-fd7l krb5kdc[27266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:32:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:32:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:32:10.097903 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38725
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:32:10.521924 27283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:32:10.522300 27283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:32:10.522487 27283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:32:10.531152 27283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:32:10.531327 27283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:32:10.531436 27283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:32:10.531534 27283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:32:10.531646 27283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:32:10.542390 27283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38725
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:32:10.544078 27283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:32:10.545902 27283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:32:10.560740 27293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:10.560796 27291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:10.560693 27290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:32:10.562347 27283 server_base.cc:1034] running on GCE node
I20260811 00:32:10.563628 27283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:32:10.565575 27283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:32:10.566866 27283 hybrid_clock.cc:648] HybridClock initialized: now 1786408330566814 us; error 48 us; skew 500 ppm
I20260811 00:32:10.567322 27283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:32:10 dist-test-slave-fd7l krb5kdc[27266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408330, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:32:10.571147 27283 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:32:10.572907 27283 webserver.cc:460] Webserver started at http://127.0.0.1:46499/ using document root <none> and password file <none>
I20260811 00:32:10.573634 27283 fs_manager.cc:359] Metadata directory not provided
I20260811 00:32:10.573746 27283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:32:10.574040 27283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:32:10.577481 27283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "d68b94f9e27247f8a02404963084b1bb"
format_stamp: "Formatted at 2026-08-11 00:32:10 on dist-test-slave-fd7l"
server_key: "14bdd5f30c141cf794b8594845a20d89"
server_key_iv: "fa080667fb9809093824e0116a135a77"
server_key_version: "encryptionkey@0"
I20260811 00:32:10.578377 27283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "d68b94f9e27247f8a02404963084b1bb"
format_stamp: "Formatted at 2026-08-11 00:32:10 on dist-test-slave-fd7l"
server_key: "14bdd5f30c141cf794b8594845a20d89"
server_key_iv: "fa080667fb9809093824e0116a135a77"
server_key_version: "encryptionkey@0"
I20260811 00:32:10.584152 27283 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:32:10.588583 27300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:32:10.590194 27283 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:32:10.590440 27283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "d68b94f9e27247f8a02404963084b1bb"
format_stamp: "Formatted at 2026-08-11 00:32:10 on dist-test-slave-fd7l"
server_key: "14bdd5f30c141cf794b8594845a20d89"
server_key_iv: "fa080667fb9809093824e0116a135a77"
server_key_version: "encryptionkey@0"
I20260811 00:32:10.590690 27283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:32:10.597680 27283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:32:10.602731 27283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:32:10.603075 27283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:32:10.623644 27283 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38725
I20260811 00:32:10.623838 27360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38725 every 8 connection(s)
I20260811 00:32:10.625423 27283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:32:10.630435 27361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:32:10.636061 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 27283
I20260811 00:32:10.636593 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:32:10.637269 2528 external_mini_cluster.cc:1428] Setting key 3e97ffd9263e36ddbe9273626f8827a3
I20260811 00:32:10.645272 27361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb: Bootstrap starting.
I20260811 00:32:10.648591 27364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:10.639823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:58209 (local address 127.0.0.1:38725)
0811 00:32:10.640824 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:10.640844 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:10.640907 (+ 63us) server_negotiation.cc:407] Connection header received
0811 00:32:10.643617 (+ 2710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:10.643656 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:10.644223 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:10.644812 (+ 589us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:10.646105 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:10.646114 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:10.646135 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:32:10.646164 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:10.646242 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:32:10.646276 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:10.646569 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:10.646728 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 00:32:10.646979 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":394,"threads_started":1}
I20260811 00:32:10.650100 27361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb: Neither blocks nor log segments found. Creating new log.
I20260811 00:32:10.651547 27361 log.cc:824] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb: Log is configured to *not* fsync() on all Append() calls
I20260811 00:32:10.654240 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:32:10.655730 27361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb: No bootstrap required, opened a new log
I20260811 00:32:10.662549 27364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:10.659841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:33267 (local address 127.0.0.1:38725)
0811 00:32:10.660155 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:10.660163 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:10.660198 (+ 35us) server_negotiation.cc:407] Connection header received
0811 00:32:10.660402 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:10.660413 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:10.660572 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:10.660737 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:10.661784 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:10.661790 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:10.661795 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:32:10.661822 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:10.661881 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:32:10.661911 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:10.662066 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:10.662117 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 00:32:10.662213 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260811 00:32:10.667485 27361 raft_consensus.cc:348] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d68b94f9e27247f8a02404963084b1bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38725 } }
I20260811 00:32:10.668188 27361 raft_consensus.cc:374] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:32:10.668360 27361 raft_consensus.cc:729] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d68b94f9e27247f8a02404963084b1bb, State: Initialized, Role: FOLLOWER
I20260811 00:32:10.669150 27361 consensus_queue.cc:260] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d68b94f9e27247f8a02404963084b1bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38725 } }
I20260811 00:32:10.669477 27361 raft_consensus.cc:388] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:32:10.669657 27361 raft_consensus.cc:482] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:32:10.669855 27361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:32:10.672272 27361 raft_consensus.cc:504] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d68b94f9e27247f8a02404963084b1bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38725 } }
I20260811 00:32:10.673089 27361 leader_election.cc:302] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d68b94f9e27247f8a02404963084b1bb; no voters:
I20260811 00:32:10.673810 27361 leader_election.cc:288] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:32:10.673943 27372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:32:10.674805 27372 raft_consensus.cc:686] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb [term 1 LEADER]: Becoming Leader. State: Replica: d68b94f9e27247f8a02404963084b1bb, State: Running, Role: LEADER
I20260811 00:32:10.675484 27372 consensus_queue.cc:237] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d68b94f9e27247f8a02404963084b1bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38725 } }
I20260811 00:32:10.676455 27361 sys_catalog.cc:564] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:32:10.678624 27373 sys_catalog.cc:455] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d68b94f9e27247f8a02404963084b1bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d68b94f9e27247f8a02404963084b1bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38725 } } }
I20260811 00:32:10.679646 27373 sys_catalog.cc:458] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:10.679397 27374 sys_catalog.cc:455] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d68b94f9e27247f8a02404963084b1bb. Latest consensus state: current_term: 1 leader_uuid: "d68b94f9e27247f8a02404963084b1bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d68b94f9e27247f8a02404963084b1bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38725 } } }
I20260811 00:32:10.679901 27374 sys_catalog.cc:458] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:10.682281 27379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:32:10.689072 27379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:32:10.701426 27379 catalog_manager.cc:1347] Generated new cluster ID: e3acc48c99214b52a41e68acf5de3929
I20260811 00:32:10.701618 27379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:32:10.739342 27379 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:32:10.740454 27379 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:32:10.755829 27379 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d68b94f9e27247f8a02404963084b1bb: Generated new TSK 0
I20260811 00:32:10.758140 27379 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:32:10.779933 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 27283
W20260811 00:32:10.781492 27356 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.241:58209
W20260811 00:32:10.782456 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 27283: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:32:10.782631 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 27283
2026-08-11T00:32:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2336 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:32:10 dist-test-slave-fd7l krb5kdc[27395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:32:10 dist-test-slave-fd7l krb5kdc[27395](info): set up 2 sockets
Aug 11 00:32:10 dist-test-slave-fd7l krb5kdc[27395](info): commencing operation
krb5kdc: starting...
W20260811 00:32:12.106817 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:32:12 dist-test-slave-fd7l krb5kdc[27395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:32:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:32:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:32:12.469457 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:32849
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:32849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:32:12.848178 27410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:32:12.848481 27410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:32:12.848676 27410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:32:12.856213 27410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:32:12.856349 27410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:32:12.856433 27410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:32:12.856593 27410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:32:12.856666 27410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:32:12.867615 27410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:32849
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:32849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:32:12.869189 27410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:32:12.870826 27410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:32:12.884799 27416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:12.885321 27421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:12.885365 27415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:32:12.886237 27410 server_base.cc:1034] running on GCE node
I20260811 00:32:12.887148 27410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:32:12.889106 27410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:32:12.890357 27410 hybrid_clock.cc:648] HybridClock initialized: now 1786408332890288 us; error 63 us; skew 500 ppm
I20260811 00:32:12.890826 27410 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:32:12 dist-test-slave-fd7l krb5kdc[27395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408332, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:32:12.900116 27410 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:32:12.902407 27410 webserver.cc:460] Webserver started at http://127.2.120.62:38917/ using document root <none> and password file <none>
I20260811 00:32:12.903272 27410 fs_manager.cc:359] Metadata directory not provided
I20260811 00:32:12.903445 27410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:32:12.903802 27410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:32:12.907342 27410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "bf5276b13b4e4495a4dd5ff5723fbff4"
format_stamp: "Formatted at 2026-08-11 00:32:12 on dist-test-slave-fd7l"
server_key: "941cfcdb701558234b7f600517d02a40"
server_key_iv: "234128cc603f88f29aac54bc82305eac"
server_key_version: "encryptionkey@0"
I20260811 00:32:12.908237 27410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "bf5276b13b4e4495a4dd5ff5723fbff4"
format_stamp: "Formatted at 2026-08-11 00:32:12 on dist-test-slave-fd7l"
server_key: "941cfcdb701558234b7f600517d02a40"
server_key_iv: "234128cc603f88f29aac54bc82305eac"
server_key_version: "encryptionkey@0"
I20260811 00:32:12.915246 27410 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.002s
I20260811 00:32:12.921428 27428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:32:12.923473 27410 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:32:12.923725 27410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "bf5276b13b4e4495a4dd5ff5723fbff4"
format_stamp: "Formatted at 2026-08-11 00:32:12 on dist-test-slave-fd7l"
server_key: "941cfcdb701558234b7f600517d02a40"
server_key_iv: "234128cc603f88f29aac54bc82305eac"
server_key_version: "encryptionkey@0"
I20260811 00:32:12.924017 27410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:32:12.961853 27410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:32:12.969391 27410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:32:12.969849 27410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:32:12.995378 27410 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:32849
I20260811 00:32:12.995419 27486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:32849 every 8 connection(s)
I20260811 00:32:12.997069 27410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:32:12.997699 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 27410
I20260811 00:32:12.997924 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:32:12.998639 2528 external_mini_cluster.cc:1428] Setting key be36d6f15a3f720961554a2f3dfa006a
I20260811 00:32:13.002447 27487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:32:13.014146 27487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4: Bootstrap starting.
Aug 11 00:32:13 dist-test-slave-fd7l krb5kdc[27395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:32:13.018316 27487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4: Neither blocks nor log segments found. Creating new log.
I20260811 00:32:13.019532 27487 log.cc:824] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4: Log is configured to *not* fsync() on all Append() calls
I20260811 00:32:13.023752 27487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4: No bootstrap required, opened a new log
I20260811 00:32:13.025137 27490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:13.000946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43761 (local address 127.2.120.62:32849)
0811 00:32:13.001838 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:13.001852 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:13.001896 (+ 44us) server_negotiation.cc:407] Connection header received
0811 00:32:13.003814 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:13.003838 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:13.004275 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:13.004743 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:13.006675 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:13.008643 (+ 1968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:13.009991 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:13.010722 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:13.010939 (+ 217us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:13.015750 (+ 4811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:13.015759 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:13.015774 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:13.015819 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:13.019332 (+ 3513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:13.020814 (+ 1482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:13.020826 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:13.020836 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:13.021135 (+ 299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:13.021782 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:13.021786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:13.021788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:13.022377 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:13.022657 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:13.023368 (+ 711us) server_negotiation.cc:299] Negotiation successful
0811 00:32:13.023865 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":393,"threads_started":1}
I20260811 00:32:13.030756 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:32:13.035928 27487 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf5276b13b4e4495a4dd5ff5723fbff4" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 32849 } }
I20260811 00:32:13.036806 27487 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:32:13.036927 27487 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf5276b13b4e4495a4dd5ff5723fbff4, State: Initialized, Role: FOLLOWER
I20260811 00:32:13.037817 27487 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf5276b13b4e4495a4dd5ff5723fbff4" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 32849 } }
I20260811 00:32:13.038174 27487 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:32:13.038357 27487 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:32:13.038602 27487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:32:13.041944 27487 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf5276b13b4e4495a4dd5ff5723fbff4" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 32849 } }
I20260811 00:32:13.042693 27487 leader_election.cc:302] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf5276b13b4e4495a4dd5ff5723fbff4; no voters:
I20260811 00:32:13.043503 27487 leader_election.cc:288] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:32:13.043672 27498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:32:13.045210 27498 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4 [term 1 LEADER]: Becoming Leader. State: Replica: bf5276b13b4e4495a4dd5ff5723fbff4, State: Running, Role: LEADER
I20260811 00:32:13.045897 27498 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf5276b13b4e4495a4dd5ff5723fbff4" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 32849 } }
I20260811 00:32:13.046324 27487 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:32:13.049003 27500 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf5276b13b4e4495a4dd5ff5723fbff4. Latest consensus state: current_term: 1 leader_uuid: "bf5276b13b4e4495a4dd5ff5723fbff4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf5276b13b4e4495a4dd5ff5723fbff4" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 32849 } } }
I20260811 00:32:13.049471 27500 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:13.050122 27499 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf5276b13b4e4495a4dd5ff5723fbff4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf5276b13b4e4495a4dd5ff5723fbff4" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 32849 } } }
I20260811 00:32:13.050483 27499 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:13.058676 27504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:32:13.059513 27490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:13.037361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57589 (local address 127.2.120.62:32849)
0811 00:32:13.037599 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:13.037609 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:13.038164 (+ 555us) server_negotiation.cc:407] Connection header received
0811 00:32:13.038395 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:13.038400 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:13.038498 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:13.038633 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:13.040672 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:13.041531 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:13.043507 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:13.044401 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:13.044689 (+ 288us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:13.048369 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:13.048382 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:13.048388 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:13.048462 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:13.053942 (+ 5480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:13.055616 (+ 1674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:13.055626 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:13.055632 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:13.055953 (+ 321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:13.056857 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:13.056867 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:13.056873 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:13.057468 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:13.057930 (+ 462us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:13.058871 (+ 941us) server_negotiation.cc:299] Negotiation successful
0811 00:32:13.059105 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 00:32:13.066061 27504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:32:13.078877 27504 catalog_manager.cc:1347] Generated new cluster ID: a91f752467b84ba8880d5bf3e0b0b7a1
I20260811 00:32:13.079020 27504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:32:13.111351 27504 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:32:13.113044 27504 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:32:13.123685 27504 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf5276b13b4e4495a4dd5ff5723fbff4: Generated new TSK 0
I20260811 00:32:13.124804 27504 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:32:13.138134 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 27410
I20260811 00:32:13.141639 27480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:32:13.253427 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 27410
W20260811 00:32:13.277382 27393 connection.cc:535] client connection to 127.2.120.62:32849 recv error: Network error: failed to read from TLS socket (remote: 127.2.120.62:32849): Connection reset by peer (error 104)
2026-08-11T00:32:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2466 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:32:13 dist-test-slave-fd7l krb5kdc[27522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:32:13 dist-test-slave-fd7l krb5kdc[27522](info): set up 2 sockets
Aug 11 00:32:13 dist-test-slave-fd7l krb5kdc[27522](info): commencing operation
krb5kdc: starting...
W20260811 00:32:14.498199 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:32:14 dist-test-slave-fd7l krb5kdc[27522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:32:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:32:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:32:14.884863 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:39505
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:44803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:39505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:32:15.308283 27535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:32:15.308881 27535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:32:15.309188 27535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:32:15.319482 27535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:32:15.319666 27535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:32:15.319721 27535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:32:15.319870 27535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:32:15.319943 27535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:32:15.333144 27535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:44803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:39505
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:39505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:32:15.334796 27535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:32:15.336712 27535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:32:15.351281 27540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:15.351358 27543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:15.351383 27541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:32:15.352339 27535 server_base.cc:1034] running on GCE node
I20260811 00:32:15.353353 27535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:32:15.355307 27535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:32:15.356786 27535 hybrid_clock.cc:648] HybridClock initialized: now 1786408335356621 us; error 168 us; skew 500 ppm
I20260811 00:32:15.357235 27535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:32:15 dist-test-slave-fd7l krb5kdc[27522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408335, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:32:15.361775 27535 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:32:15.363947 27535 webserver.cc:460] Webserver started at http://127.2.120.62:40503/ using document root <none> and password file <none>
I20260811 00:32:15.364990 27535 fs_manager.cc:359] Metadata directory not provided
I20260811 00:32:15.365181 27535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:32:15.365578 27535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:32:15.369477 27535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "cd2d9332d1f64d7c92634ef516b34961"
format_stamp: "Formatted at 2026-08-11 00:32:15 on dist-test-slave-fd7l"
server_key: "6fbad709218226ba86146f3953c56b80"
server_key_iv: "7487cc410d1284145442f64d4984f4bd"
server_key_version: "encryptionkey@0"
I20260811 00:32:15.370501 27535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "cd2d9332d1f64d7c92634ef516b34961"
format_stamp: "Formatted at 2026-08-11 00:32:15 on dist-test-slave-fd7l"
server_key: "6fbad709218226ba86146f3953c56b80"
server_key_iv: "7487cc410d1284145442f64d4984f4bd"
server_key_version: "encryptionkey@0"
I20260811 00:32:15.377487 27535 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:32:15.382278 27553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:32:15.384132 27535 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:32:15.384388 27535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "cd2d9332d1f64d7c92634ef516b34961"
format_stamp: "Formatted at 2026-08-11 00:32:15 on dist-test-slave-fd7l"
server_key: "6fbad709218226ba86146f3953c56b80"
server_key_iv: "7487cc410d1284145442f64d4984f4bd"
server_key_version: "encryptionkey@0"
I20260811 00:32:15.384739 27535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:32:15.411441 27535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:32:15.416661 27535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:32:15.416985 27535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:32:15.439206 27535 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:39505
I20260811 00:32:15.439258 27614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:39505 every 8 connection(s)
I20260811 00:32:15.440773 27535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:32:15.445628 27615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:32:15.446355 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 27535
I20260811 00:32:15.446563 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:32:15.447438 2528 external_mini_cluster.cc:1428] Setting key 4590fd230ba80c90ac3e451379ef41aa
I20260811 00:32:15.461133 27615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961: Bootstrap starting.
I20260811 00:32:15.464233 27618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:15.449816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35303 (local address 127.2.120.62:39505)
0811 00:32:15.451796 (+ 1980us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:15.451810 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:15.451861 (+ 51us) server_negotiation.cc:407] Connection header received
0811 00:32:15.453870 (+ 2009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:15.453905 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:15.454427 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:15.454920 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:15.455523 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:15.456974 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:15.458414 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:15.459193 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:15.459406 (+ 213us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:15.460932 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:15.460941 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:15.460958 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:32:15.461608 (+ 650us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:15.461703 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:32:15.461732 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:15.461916 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:15.462232 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 00:32:15.462449 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1685,"thread_start_us":1372,"threads_started":1}
I20260811 00:32:15.465917 27615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961: Neither blocks nor log segments found. Creating new log.
I20260811 00:32:15.468549 27615 log.cc:824] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961: Log is configured to *not* fsync() on all Append() calls
I20260811 00:32:15.471843 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:32:15.472702 27615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961: No bootstrap required, opened a new log
I20260811 00:32:15.482096 27615 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2d9332d1f64d7c92634ef516b34961" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39505 } }
I20260811 00:32:15.482688 27615 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:32:15.482841 27615 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd2d9332d1f64d7c92634ef516b34961, State: Initialized, Role: FOLLOWER
I20260811 00:32:15.483587 27615 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2d9332d1f64d7c92634ef516b34961" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39505 } }
I20260811 00:32:15.483903 27615 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:32:15.484093 27615 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:32:15.484304 27615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:32:15.484978 27618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:15.476987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55533 (local address 127.2.120.62:39505)
0811 00:32:15.477184 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:15.477189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:15.477751 (+ 562us) server_negotiation.cc:407] Connection header received
0811 00:32:15.478010 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:15.478016 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:15.478101 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:15.478243 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:15.478806 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:15.479655 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:15.481727 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:15.482341 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:15.482556 (+ 215us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:15.484053 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:15.484062 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:15.484068 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:32:15.484097 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:15.484155 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:32:15.484174 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:15.484334 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:15.484602 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 00:32:15.484669 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 00:32:15.487774 27615 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2d9332d1f64d7c92634ef516b34961" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39505 } }
I20260811 00:32:15.488453 27615 leader_election.cc:302] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd2d9332d1f64d7c92634ef516b34961; no voters:
I20260811 00:32:15.489269 27615 leader_election.cc:288] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:32:15.489439 27625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:32:15.491434 27615 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:32:15.496464 27625 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961 [term 1 LEADER]: Becoming Leader. State: Replica: cd2d9332d1f64d7c92634ef516b34961, State: Running, Role: LEADER
I20260811 00:32:15.497424 27625 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2d9332d1f64d7c92634ef516b34961" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39505 } }
I20260811 00:32:15.501199 27627 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd2d9332d1f64d7c92634ef516b34961" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2d9332d1f64d7c92634ef516b34961" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39505 } } }
I20260811 00:32:15.501792 27627 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:15.504588 27642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:32:15.504333 27628 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd2d9332d1f64d7c92634ef516b34961. Latest consensus state: current_term: 1 leader_uuid: "cd2d9332d1f64d7c92634ef516b34961" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2d9332d1f64d7c92634ef516b34961" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39505 } } }
I20260811 00:32:15.508607 27628 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:15.509661 27642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:32:15.520397 27642 catalog_manager.cc:1347] Generated new cluster ID: 437d8655702440079beaa958709095a8
I20260811 00:32:15.520560 27642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:32:15.547369 27642 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:32:15.548817 27642 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:32:15.560395 27642 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd2d9332d1f64d7c92634ef516b34961: Generated new TSK 0
I20260811 00:32:15.561501 27642 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:32:15.573309 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 27535
W20260811 00:32:15.575047 27609 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35303
W20260811 00:32:15.576046 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 27535: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:32:15.576485 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 27535
W20260811 00:32:15.602581 27520 connection.cc:535] client connection to 127.2.120.62:39505 recv error: Network error: failed to read from TLS socket (remote: 127.2.120.62:39505): Connection reset by peer (error 104)
2026-08-11T00:32:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2325 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:32:15 dist-test-slave-fd7l krb5kdc[27649](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:32:15 dist-test-slave-fd7l krb5kdc[27649](info): set up 2 sockets
Aug 11 00:32:15 dist-test-slave-fd7l krb5kdc[27649](info): commencing operation
krb5kdc: starting...
W20260811 00:32:16.852588 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:32:17 dist-test-slave-fd7l krb5kdc[27649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:32:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:32:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:32:17.216632 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:38865
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:42873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:38865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:32:17.611147 27664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:32:17.611507 27664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:32:17.611692 27664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:32:17.619606 27664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:32:17.619788 27664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:32:17.619908 27664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:32:17.620018 27664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:32:17.620107 27664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:32:17.631085 27664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:42873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:38865
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:38865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:32:17.632658 27664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:32:17.634130 27664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:32:17.648667 27670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:17.648605 27672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:17.648732 27669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:32:17.650077 27664 server_base.cc:1034] running on GCE node
I20260811 00:32:17.650985 27664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:32:17.652803 27664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:32:17.654070 27664 hybrid_clock.cc:648] HybridClock initialized: now 1786408337654024 us; error 44 us; skew 500 ppm
I20260811 00:32:17.654520 27664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:32:17 dist-test-slave-fd7l krb5kdc[27649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408337, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:32:17.658349 27664 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:32:17.661006 27664 webserver.cc:460] Webserver started at http://127.2.120.62:45683/ using document root <none> and password file <none>
I20260811 00:32:17.661880 27664 fs_manager.cc:359] Metadata directory not provided
I20260811 00:32:17.662042 27664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:32:17.662401 27664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:32:17.665827 27664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "ed54b4210b704fd78aca4a1fcf558918"
format_stamp: "Formatted at 2026-08-11 00:32:17 on dist-test-slave-fd7l"
server_key: "cb739ad84e34fcb6c9ffd2e0e100d059"
server_key_iv: "89c74926fe31ea3a6f608dc03b5c914f"
server_key_version: "encryptionkey@0"
I20260811 00:32:17.666664 27664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "ed54b4210b704fd78aca4a1fcf558918"
format_stamp: "Formatted at 2026-08-11 00:32:17 on dist-test-slave-fd7l"
server_key: "cb739ad84e34fcb6c9ffd2e0e100d059"
server_key_iv: "89c74926fe31ea3a6f608dc03b5c914f"
server_key_version: "encryptionkey@0"
I20260811 00:32:17.672744 27664 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 00:32:17.677546 27679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:32:17.679272 27664 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:32:17.679524 27664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "ed54b4210b704fd78aca4a1fcf558918"
format_stamp: "Formatted at 2026-08-11 00:32:17 on dist-test-slave-fd7l"
server_key: "cb739ad84e34fcb6c9ffd2e0e100d059"
server_key_iv: "89c74926fe31ea3a6f608dc03b5c914f"
server_key_version: "encryptionkey@0"
I20260811 00:32:17.679785 27664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:32:17.686228 27664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:32:17.691371 27664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:32:17.691762 27664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:32:17.713836 27664 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:38865
I20260811 00:32:17.713971 27736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:38865 every 8 connection(s)
I20260811 00:32:17.715592 27664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:32:17.719722 27737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:32:17.724663 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 27664
I20260811 00:32:17.724911 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:32:17.725414 2528 external_mini_cluster.cc:1428] Setting key e159b0f2641ed69ce3d5f8cacb2afa73
I20260811 00:32:17.732383 27737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918: Bootstrap starting.
I20260811 00:32:17.734825 27740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:17.727502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41837 (local address 127.2.120.62:38865)
0811 00:32:17.728606 (+ 1104us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:17.728630 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:17.728691 (+ 61us) server_negotiation.cc:407] Connection header received
0811 00:32:17.730727 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:17.730758 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:17.731195 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:17.731612 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:17.733033 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:17.733042 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:17.733056 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:32:17.733080 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:17.733156 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:32:17.733180 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:17.733386 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:17.733507 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 00:32:17.733735 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":751,"thread_start_us":477,"threads_started":1}
I20260811 00:32:17.736321 27737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918: Neither blocks nor log segments found. Creating new log.
I20260811 00:32:17.737594 27737 log.cc:824] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918: Log is configured to *not* fsync() on all Append() calls
I20260811 00:32:17.739496 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:32:17.741156 27737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918: No bootstrap required, opened a new log
I20260811 00:32:17.747478 27740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:17.744701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41191 (local address 127.2.120.62:38865)
0811 00:32:17.744934 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:17.744942 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:17.745356 (+ 414us) server_negotiation.cc:407] Connection header received
0811 00:32:17.745571 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:17.745580 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:17.745640 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:17.745753 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:17.746762 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:17.746768 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:17.746773 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:32:17.746795 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:17.746849 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:32:17.746868 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:17.747010 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:17.747117 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 00:32:17.747184 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 00:32:17.750622 27737 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed54b4210b704fd78aca4a1fcf558918" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 38865 } }
I20260811 00:32:17.751170 27737 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:32:17.751309 27737 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed54b4210b704fd78aca4a1fcf558918, State: Initialized, Role: FOLLOWER
I20260811 00:32:17.751936 27737 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed54b4210b704fd78aca4a1fcf558918" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 38865 } }
I20260811 00:32:17.752154 27737 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:32:17.752257 27737 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:32:17.752386 27737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:32:17.754750 27737 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed54b4210b704fd78aca4a1fcf558918" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 38865 } }
I20260811 00:32:17.755277 27737 leader_election.cc:302] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed54b4210b704fd78aca4a1fcf558918; no voters:
I20260811 00:32:17.755903 27737 leader_election.cc:288] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:32:17.756093 27748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:32:17.757175 27748 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918 [term 1 LEADER]: Becoming Leader. State: Replica: ed54b4210b704fd78aca4a1fcf558918, State: Running, Role: LEADER
I20260811 00:32:17.757792 27748 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed54b4210b704fd78aca4a1fcf558918" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 38865 } }
I20260811 00:32:17.758246 27737 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:32:17.761272 27750 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed54b4210b704fd78aca4a1fcf558918. Latest consensus state: current_term: 1 leader_uuid: "ed54b4210b704fd78aca4a1fcf558918" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed54b4210b704fd78aca4a1fcf558918" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 38865 } } }
I20260811 00:32:17.761384 27749 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed54b4210b704fd78aca4a1fcf558918" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed54b4210b704fd78aca4a1fcf558918" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 38865 } } }
I20260811 00:32:17.761885 27750 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:17.761938 27749 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:17.765282 27756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:32:17.770500 27756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:32:17.781075 27756 catalog_manager.cc:1347] Generated new cluster ID: d68f5a13da2b464585bb6e52ace6b90f
I20260811 00:32:17.781224 27756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:32:17.808343 27756 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:32:17.808866 27756 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:32:17.815629 27756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed54b4210b704fd78aca4a1fcf558918: Generated new TSK 0
I20260811 00:32:17.816634 27756 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:32:17.825840 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 27664
W20260811 00:32:17.827458 27732 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41837
W20260811 00:32:17.828341 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 27664: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:32:17.828553 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 27664
2026-08-11T00:32:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2250 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:32:17 dist-test-slave-fd7l krb5kdc[27769](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:32:17 dist-test-slave-fd7l krb5kdc[27769](info): set up 2 sockets
Aug 11 00:32:17 dist-test-slave-fd7l krb5kdc[27769](info): commencing operation
krb5kdc: starting...
W20260811 00:32:19.105960 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:32:19 dist-test-slave-fd7l krb5kdc[27769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:32:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:32:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:32:19.477514 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:38683
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:37589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:38683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:32:19.875408 27782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:32:19.875775 27782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:32:19.875873 27782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:32:19.883698 27782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:32:19.884104 27782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:32:19.884392 27782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:32:19.884663 27782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:32:19.884765 27782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:32:19.895490 27782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:38683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:38683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:32:19.897054 27782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:32:19.898564 27782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:32:19.912657 27789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:19.912686 27788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:19.912757 27791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:32:19.913489 27782 server_base.cc:1034] running on GCE node
I20260811 00:32:19.914433 27782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:32:19.916177 27782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:32:19.917407 27782 hybrid_clock.cc:648] HybridClock initialized: now 1786408339917338 us; error 62 us; skew 500 ppm
I20260811 00:32:19.917827 27782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:32:19 dist-test-slave-fd7l krb5kdc[27769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408339, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:32:19.921435 27782 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:32:19.923224 27782 webserver.cc:460] Webserver started at http://127.2.120.62:44629/ using document root <none> and password file <none>
I20260811 00:32:19.924041 27782 fs_manager.cc:359] Metadata directory not provided
I20260811 00:32:19.924225 27782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:32:19.924643 27782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:32:19.928202 27782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "41f79a7eed50421881c5d7efb479f1af"
format_stamp: "Formatted at 2026-08-11 00:32:19 on dist-test-slave-fd7l"
server_key: "b55fd82e5b2ee60ad8535b245ebeb646"
server_key_iv: "265e8b3ca3d98872e65a66fea1b93298"
server_key_version: "encryptionkey@0"
I20260811 00:32:19.929226 27782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "41f79a7eed50421881c5d7efb479f1af"
format_stamp: "Formatted at 2026-08-11 00:32:19 on dist-test-slave-fd7l"
server_key: "b55fd82e5b2ee60ad8535b245ebeb646"
server_key_iv: "265e8b3ca3d98872e65a66fea1b93298"
server_key_version: "encryptionkey@0"
I20260811 00:32:19.936087 27782 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 00:32:19.940800 27799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:32:19.943078 27782 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:32:19.943379 27782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "41f79a7eed50421881c5d7efb479f1af"
format_stamp: "Formatted at 2026-08-11 00:32:19 on dist-test-slave-fd7l"
server_key: "b55fd82e5b2ee60ad8535b245ebeb646"
server_key_iv: "265e8b3ca3d98872e65a66fea1b93298"
server_key_version: "encryptionkey@0"
I20260811 00:32:19.943663 27782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:32:19.960136 27782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:32:19.965232 27782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:32:19.965574 27782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:32:19.988081 27782 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:38683
I20260811 00:32:19.988133 27853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:38683 every 8 connection(s)
I20260811 00:32:19.989749 27782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:32:19.994184 27854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:32:19.997618 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 27782
I20260811 00:32:19.997941 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:32:19.998559 2528 external_mini_cluster.cc:1428] Setting key 9f75f2047104cc20f279710e74949c6c
I20260811 00:32:20.009167 27854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af: Bootstrap starting.
I20260811 00:32:20.013746 27854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af: Neither blocks nor log segments found. Creating new log.
I20260811 00:32:20.015373 27854 log.cc:824] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:32:20 dist-test-slave-fd7l krb5kdc[27769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:32:20.019062 27854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af: No bootstrap required, opened a new log
I20260811 00:32:20.032357 27854 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41f79a7eed50421881c5d7efb479f1af" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 38683 } }
I20260811 00:32:20.033555 27854 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:32:20.033771 27854 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41f79a7eed50421881c5d7efb479f1af, State: Initialized, Role: FOLLOWER
I20260811 00:32:20.034521 27857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:20.000996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37409 (local address 127.2.120.62:38683)
0811 00:32:20.001928 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:20.001941 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:20.001988 (+ 47us) server_negotiation.cc:407] Connection header received
0811 00:32:20.004032 (+ 2044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:20.004064 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:20.004963 (+ 899us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:20.005606 (+ 643us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:20.007803 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:20.009781 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:20.012006 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:20.013065 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:20.013259 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:20.018668 (+ 5409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:20.018678 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:20.018697 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:20.018740 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:20.022910 (+ 4170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:20.024145 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:20.024165 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:20.024181 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:20.024798 (+ 617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:20.025867 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:20.025877 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:20.025880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:20.027713 (+ 1833us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:20.028738 (+ 1025us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:20.030164 (+ 1426us) server_negotiation.cc:299] Negotiation successful
0811 00:32:20.031391 (+ 1227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":449,"threads_started":1}
I20260811 00:32:20.035331 27854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41f79a7eed50421881c5d7efb479f1af" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 38683 } }
I20260811 00:32:20.036237 27854 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:32:20.036589 27854 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:32:20.036908 27854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:32:20.041535 27854 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41f79a7eed50421881c5d7efb479f1af" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 38683 } }
I20260811 00:32:20.042917 27854 leader_election.cc:302] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41f79a7eed50421881c5d7efb479f1af; no voters:
I20260811 00:32:20.044822 27854 leader_election.cc:288] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:32:20.044950 27859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:32:20.045890 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:32:20.049455 27859 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af [term 1 LEADER]: Becoming Leader. State: Replica: 41f79a7eed50421881c5d7efb479f1af, State: Running, Role: LEADER
I20260811 00:32:20.050074 27854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:32:20.050980 27859 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41f79a7eed50421881c5d7efb479f1af" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 38683 } }
I20260811 00:32:20.059669 27862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41f79a7eed50421881c5d7efb479f1af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41f79a7eed50421881c5d7efb479f1af" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 38683 } } }
I20260811 00:32:20.062472 27862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:20.061681 27860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41f79a7eed50421881c5d7efb479f1af. Latest consensus state: current_term: 1 leader_uuid: "41f79a7eed50421881c5d7efb479f1af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41f79a7eed50421881c5d7efb479f1af" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 38683 } } }
I20260811 00:32:20.063031 27860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:20.078788 27875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:32:20.088366 27875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:32:20.092478 27857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:20.059487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44425 (local address 127.2.120.62:38683)
0811 00:32:20.059964 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:20.059985 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:20.062414 (+ 2429us) server_negotiation.cc:407] Connection header received
0811 00:32:20.063035 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:20.063046 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:20.063211 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:20.063341 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:20.067239 (+ 3898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:20.068891 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:20.072566 (+ 3675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:20.073915 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:20.074211 (+ 296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:20.081509 (+ 7298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:20.081519 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:20.081525 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:20.081598 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:20.086752 (+ 5154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:20.087776 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:20.087783 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:20.087787 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:20.087997 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:20.088497 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:20.088502 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:20.088505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:20.089104 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:20.090983 (+ 1879us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:20.091221 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 00:32:20.091597 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260811 00:32:20.104902 27875 catalog_manager.cc:1347] Generated new cluster ID: 63ac2a26550145bd9d6a68e0a9eb3b08
I20260811 00:32:20.105065 27875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:32:20.133606 27875 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:32:20.135962 27875 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:32:20.148491 27875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41f79a7eed50421881c5d7efb479f1af: Generated new TSK 0
I20260811 00:32:20.149860 27875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:32:20.162489 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 27782
I20260811 00:32:20.165073 27848 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:32:20.270928 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 27782
2026-08-11T00:32:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:32:20 dist-test-slave-fd7l krb5kdc[27892](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:32:20 dist-test-slave-fd7l krb5kdc[27892](info): set up 2 sockets
Aug 11 00:32:20 dist-test-slave-fd7l krb5kdc[27892](info): commencing operation
krb5kdc: starting...
W20260811 00:32:21.568261 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:32:21 dist-test-slave-fd7l krb5kdc[27892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:32:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:32:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:32:21.932484 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:42833
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:43449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:42833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:32:22.317734 27907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:32:22.318076 27907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:32:22.318179 27907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:32:22.325857 27907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:32:22.326002 27907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:32:22.326104 27907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:32:22.326228 27907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:32:22.326310 27907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:32:22.337836 27907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:43449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:42833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:42833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:32:22.339425 27907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:32:22.341228 27907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:32:22.354657 27913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:22.354764 27915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:22.354691 27912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:32:22.356035 27907 server_base.cc:1034] running on GCE node
I20260811 00:32:22.357033 27907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:32:22.358598 27907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:32:22.359859 27907 hybrid_clock.cc:648] HybridClock initialized: now 1786408342359810 us; error 39 us; skew 500 ppm
I20260811 00:32:22.360280 27907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:32:22 dist-test-slave-fd7l krb5kdc[27892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408342, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:32:22.364665 27907 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:32:22.366840 27907 webserver.cc:460] Webserver started at http://127.2.120.62:40255/ using document root <none> and password file <none>
I20260811 00:32:22.367625 27907 fs_manager.cc:359] Metadata directory not provided
I20260811 00:32:22.367748 27907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:32:22.368023 27907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:32:22.371443 27907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "43c227bf4d494a58a6f212c6414cc113"
format_stamp: "Formatted at 2026-08-11 00:32:22 on dist-test-slave-fd7l"
server_key: "45808c84b78d2a35e398bf7261f973e6"
server_key_iv: "cac4c667cb31c3949f05d9b85e1968b8"
server_key_version: "encryptionkey@0"
I20260811 00:32:22.372259 27907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "43c227bf4d494a58a6f212c6414cc113"
format_stamp: "Formatted at 2026-08-11 00:32:22 on dist-test-slave-fd7l"
server_key: "45808c84b78d2a35e398bf7261f973e6"
server_key_iv: "cac4c667cb31c3949f05d9b85e1968b8"
server_key_version: "encryptionkey@0"
I20260811 00:32:22.377959 27907 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 00:32:22.381642 27922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:32:22.383065 27907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:32:22.383275 27907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "43c227bf4d494a58a6f212c6414cc113"
format_stamp: "Formatted at 2026-08-11 00:32:22 on dist-test-slave-fd7l"
server_key: "45808c84b78d2a35e398bf7261f973e6"
server_key_iv: "cac4c667cb31c3949f05d9b85e1968b8"
server_key_version: "encryptionkey@0"
I20260811 00:32:22.383539 27907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:32:22.403247 27907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:32:22.408776 27907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:32:22.409476 27907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:32:22.431051 27907 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:42833
I20260811 00:32:22.431073 27982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:42833 every 8 connection(s)
I20260811 00:32:22.432561 27907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:32:22.436470 27983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:32:22.440953 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 27907
I20260811 00:32:22.441154 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:32:22.441748 2528 external_mini_cluster.cc:1428] Setting key 6faaa6ae9da7001fc9b295584bd359cc
I20260811 00:32:22.450591 27983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113: Bootstrap starting.
I20260811 00:32:22.454309 27983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113: Neither blocks nor log segments found. Creating new log.
I20260811 00:32:22.455698 27983 log.cc:824] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113: Log is configured to *not* fsync() on all Append() calls
I20260811 00:32:22.458673 27986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:22.443787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38801 (local address 127.2.120.62:42833)
0811 00:32:22.444753 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:22.444769 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:22.444813 (+ 44us) server_negotiation.cc:407] Connection header received
0811 00:32:22.446575 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:22.446605 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:22.447116 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:22.447594 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:22.448493 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:22.449991 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:22.452742 (+ 2751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:22.453675 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:22.453950 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:22.455522 (+ 1572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:22.455535 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:22.455552 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:32:22.456258 (+ 706us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:22.456358 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:32:22.456392 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:22.456585 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:22.456894 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 00:32:22.457256 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":398,"threads_started":1}
I20260811 00:32:22.459619 27983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113: No bootstrap required, opened a new log
I20260811 00:32:22.463091 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:32:22.468937 27983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c227bf4d494a58a6f212c6414cc113" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42833 } }
I20260811 00:32:22.469461 27983 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:32:22.469607 27983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43c227bf4d494a58a6f212c6414cc113, State: Initialized, Role: FOLLOWER
I20260811 00:32:22.470342 27983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c227bf4d494a58a6f212c6414cc113" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42833 } }
I20260811 00:32:22.470641 27983 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:32:22.470816 27983 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:32:22.470996 27983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:32:22.474213 27983 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c227bf4d494a58a6f212c6414cc113" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42833 } }
I20260811 00:32:22.474956 27986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:22.467748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49049 (local address 127.2.120.62:42833)
0811 00:32:22.467948 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:22.467955 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:22.468362 (+ 407us) server_negotiation.cc:407] Connection header received
0811 00:32:22.468552 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:22.468559 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:22.468646 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:22.468758 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:22.469279 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:22.470373 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:22.471946 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:22.472709 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:22.472861 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:22.474099 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:22.474105 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:22.474109 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:32:22.474134 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:22.474186 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:32:22.474215 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:22.474318 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:22.474534 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 00:32:22.474643 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:32:22.474931 27983 leader_election.cc:302] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43c227bf4d494a58a6f212c6414cc113; no voters:
I20260811 00:32:22.476476 27983 leader_election.cc:288] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:32:22.476835 27994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:32:22.480635 27994 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113 [term 1 LEADER]: Becoming Leader. State: Replica: 43c227bf4d494a58a6f212c6414cc113, State: Running, Role: LEADER
I20260811 00:32:22.482161 27994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c227bf4d494a58a6f212c6414cc113" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42833 } }
I20260811 00:32:22.482955 27983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:32:22.491657 27995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43c227bf4d494a58a6f212c6414cc113. Latest consensus state: current_term: 1 leader_uuid: "43c227bf4d494a58a6f212c6414cc113" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c227bf4d494a58a6f212c6414cc113" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42833 } } }
I20260811 00:32:22.491683 27996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43c227bf4d494a58a6f212c6414cc113" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c227bf4d494a58a6f212c6414cc113" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 42833 } } }
I20260811 00:32:22.493427 27995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:22.493544 27996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:22.502192 28003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:32:22.513844 28003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:32:22.530298 28003 catalog_manager.cc:1347] Generated new cluster ID: 4cd850e8e3e24c2888e96f2b4c2fae90
I20260811 00:32:22.530448 28003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:32:22.575610 28003 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:32:22.577679 28003 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:32:22.591137 28003 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43c227bf4d494a58a6f212c6414cc113: Generated new TSK 0
I20260811 00:32:22.592398 28003 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:32:22.612118 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 27907
W20260811 00:32:22.613513 27975 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38801
W20260811 00:32:22.614498 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 27907: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:32:22.614701 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 27907
2026-08-11T00:32:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2345 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:32:22 dist-test-slave-fd7l krb5kdc[28018](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:32:22 dist-test-slave-fd7l krb5kdc[28018](info): set up 2 sockets
Aug 11 00:32:22 dist-test-slave-fd7l krb5kdc[28018](info): commencing operation
krb5kdc: starting...
W20260811 00:32:23.912722 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:32:24 dist-test-slave-fd7l krb5kdc[28018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:32:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:32:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:32:24.279749 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:39343
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:37533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:39343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:32:24.694674 28031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:32:24.695099 28031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:32:24.695297 28031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:32:24.703153 28031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:32:24.703378 28031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:32:24.703472 28031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:32:24.703599 28031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:32:24.703707 28031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:32:24.714507 28031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:37533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:39343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:39343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:32:24.716204 28031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:32:24.717689 28031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:32:24.731055 28037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:24.730958 28040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:24.731025 28038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:32:24.735682 28031 server_base.cc:1034] running on GCE node
I20260811 00:32:24.736862 28031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:32:24.738703 28031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:32:24.739995 28031 hybrid_clock.cc:648] HybridClock initialized: now 1786408344739956 us; error 65 us; skew 500 ppm
I20260811 00:32:24.740442 28031 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:32:24 dist-test-slave-fd7l krb5kdc[28018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408344, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:32:24.744280 28031 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:32:24.747136 28031 webserver.cc:460] Webserver started at http://127.2.120.62:34095/ using document root <none> and password file <none>
I20260811 00:32:24.748030 28031 fs_manager.cc:359] Metadata directory not provided
I20260811 00:32:24.748210 28031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:32:24.748656 28031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:32:24.752208 28031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "01292650f9144eac8854feabf93c3b72"
format_stamp: "Formatted at 2026-08-11 00:32:24 on dist-test-slave-fd7l"
server_key: "c94204bf7c2a096c35843d8f284ea62a"
server_key_iv: "c45f579cde1ad90e66c80690b7e7ecbd"
server_key_version: "encryptionkey@0"
I20260811 00:32:24.753266 28031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "01292650f9144eac8854feabf93c3b72"
format_stamp: "Formatted at 2026-08-11 00:32:24 on dist-test-slave-fd7l"
server_key: "c94204bf7c2a096c35843d8f284ea62a"
server_key_iv: "c45f579cde1ad90e66c80690b7e7ecbd"
server_key_version: "encryptionkey@0"
I20260811 00:32:24.760231 28031 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 00:32:24.765074 28047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:32:24.766770 28031 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:32:24.767032 28031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "01292650f9144eac8854feabf93c3b72"
format_stamp: "Formatted at 2026-08-11 00:32:24 on dist-test-slave-fd7l"
server_key: "c94204bf7c2a096c35843d8f284ea62a"
server_key_iv: "c45f579cde1ad90e66c80690b7e7ecbd"
server_key_version: "encryptionkey@0"
I20260811 00:32:24.767284 28031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:32:24.773806 28031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:32:24.778657 28031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:32:24.779014 28031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:32:24.799861 28031 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:39343
I20260811 00:32:24.799961 28105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:39343 every 8 connection(s)
I20260811 00:32:24.801532 28031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:32:24.805928 28107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:32:24.807634 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 28031
I20260811 00:32:24.807852 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:32:24.808396 2528 external_mini_cluster.cc:1428] Setting key e3682e95560023461fae17a502648c00
I20260811 00:32:24.817392 28111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:24.810520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44879 (local address 127.2.120.62:39343)
0811 00:32:24.811415 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:24.811431 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:24.811495 (+ 64us) server_negotiation.cc:407] Connection header received
0811 00:32:24.813343 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:24.813378 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:24.813906 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:24.814392 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:24.815354 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:24.815363 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:24.815380 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:32:24.815398 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:24.815467 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:32:24.815512 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:24.815743 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:24.815874 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 00:32:24.816078 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":405,"threads_started":1}
I20260811 00:32:24.821134 28107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72: Bootstrap starting.
I20260811 00:32:24.821425 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:32:24.824512 28107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72: Neither blocks nor log segments found. Creating new log.
I20260811 00:32:24.825692 28107 log.cc:824] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72: Log is configured to *not* fsync() on all Append() calls
I20260811 00:32:24.829174 28107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72: No bootstrap required, opened a new log
I20260811 00:32:24.829742 28111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:24.826547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48191 (local address 127.2.120.62:39343)
0811 00:32:24.826770 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:24.826777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:24.827073 (+ 296us) server_negotiation.cc:407] Connection header received
0811 00:32:24.827280 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:24.827290 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:24.827339 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:24.827465 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:24.829004 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:24.829017 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:24.829021 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:32:24.829038 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:24.829081 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:32:24.829097 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:24.829238 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:24.829377 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 00:32:24.829447 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 00:32:24.837728 28107 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01292650f9144eac8854feabf93c3b72" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39343 } }
I20260811 00:32:24.838296 28107 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:32:24.838435 28107 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01292650f9144eac8854feabf93c3b72, State: Initialized, Role: FOLLOWER
I20260811 00:32:24.839076 28107 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01292650f9144eac8854feabf93c3b72" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39343 } }
I20260811 00:32:24.839319 28107 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:32:24.839488 28107 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:32:24.839675 28107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:32:24.842344 28107 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01292650f9144eac8854feabf93c3b72" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39343 } }
I20260811 00:32:24.843294 28107 leader_election.cc:302] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01292650f9144eac8854feabf93c3b72; no voters:
I20260811 00:32:24.844245 28107 leader_election.cc:288] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:32:24.844790 28118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:32:24.845742 28118 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72 [term 1 LEADER]: Becoming Leader. State: Replica: 01292650f9144eac8854feabf93c3b72, State: Running, Role: LEADER
I20260811 00:32:24.846414 28118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01292650f9144eac8854feabf93c3b72" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39343 } }
I20260811 00:32:24.847294 28107 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:32:24.849819 28120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01292650f9144eac8854feabf93c3b72. Latest consensus state: current_term: 1 leader_uuid: "01292650f9144eac8854feabf93c3b72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01292650f9144eac8854feabf93c3b72" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39343 } } }
I20260811 00:32:24.849932 28119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01292650f9144eac8854feabf93c3b72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01292650f9144eac8854feabf93c3b72" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 39343 } } }
I20260811 00:32:24.850281 28120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:24.850332 28119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:24.853433 28123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:32:24.859620 28123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:32:24.871439 28123 catalog_manager.cc:1347] Generated new cluster ID: 2478fc5c94694e3a8f745f0f4d96af79
I20260811 00:32:24.871623 28123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:32:24.894623 28123 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:32:24.895205 28123 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:32:24.906708 28123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01292650f9144eac8854feabf93c3b72: Generated new TSK 0
I20260811 00:32:24.908047 28123 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:32:24.921972 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 28031
W20260811 00:32:24.923244 28100 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44879
W20260811 00:32:24.924165 2528 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 28031: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:32:24.924350 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 28031
2026-08-11T00:32:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2306 ms)
[----------] 18 tests from AuthTokenIssuingTest (42256 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:32:25 dist-test-slave-fd7l krb5kdc[28140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:32:25 dist-test-slave-fd7l krb5kdc[28140](info): set up 2 sockets
Aug 11 00:32:25 dist-test-slave-fd7l krb5kdc[28140](info): commencing operation
krb5kdc: starting...
W20260811 00:32:26.212931 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.003s sys 0.060s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:32:26 dist-test-slave-fd7l krb5kdc[28140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:32:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:32:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:32:26.568926 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:37349
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:40597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:37349,127.2.120.61:33993,127.2.120.60:37335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:32:26.977298 28154 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 00:32:26.977694 28154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:32:26.977825 28154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:32:26.977969 28154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:32:26.985467 28154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:32:26.985628 28154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:32:26.985728 28154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:32:26.985816 28154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:32:26.985934 28154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:32:26.996381 28154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:40597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:37349,127.2.120.61:33993,127.2.120.60:37335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:37349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:32:26.997885 28154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:32:26.999310 28154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:32:27.013559 28159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:27.013523 28163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:27.013121 28160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:32:27.014328 28154 server_base.cc:1034] running on GCE node
I20260811 00:32:27.015304 28154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:32:27.017145 28154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:32:27.018487 28154 hybrid_clock.cc:648] HybridClock initialized: now 1786408347018402 us; error 50 us; skew 500 ppm
I20260811 00:32:27.019177 28154 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:32:27 dist-test-slave-fd7l krb5kdc[28140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408347, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:32:27.023444 28154 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:32:27.025291 28154 webserver.cc:460] Webserver started at http://127.2.120.62:43977/ using document root <none> and password file <none>
I20260811 00:32:27.026091 28154 fs_manager.cc:359] Metadata directory not provided
I20260811 00:32:27.026199 28154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:32:27.026505 28154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:32:27.030321 28154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "c461f11505e0407cacf14cd7a6a51a7b"
format_stamp: "Formatted at 2026-08-11 00:32:27 on dist-test-slave-fd7l"
server_key: "7cb0c3fffec9cae4be53807b61b76b4c"
server_key_iv: "d3951958f773dcc741396ddddc4e7be1"
server_key_version: "encryptionkey@0"
I20260811 00:32:27.031282 28154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "c461f11505e0407cacf14cd7a6a51a7b"
format_stamp: "Formatted at 2026-08-11 00:32:27 on dist-test-slave-fd7l"
server_key: "7cb0c3fffec9cae4be53807b61b76b4c"
server_key_iv: "d3951958f773dcc741396ddddc4e7be1"
server_key_version: "encryptionkey@0"
I20260811 00:32:27.037549 28154 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 00:32:27.041613 28172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:32:27.043530 28154 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:32:27.043798 28154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "c461f11505e0407cacf14cd7a6a51a7b"
format_stamp: "Formatted at 2026-08-11 00:32:27 on dist-test-slave-fd7l"
server_key: "7cb0c3fffec9cae4be53807b61b76b4c"
server_key_iv: "d3951958f773dcc741396ddddc4e7be1"
server_key_version: "encryptionkey@0"
I20260811 00:32:27.044073 28154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:32:27.065614 28154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:32:27.070732 28154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:32:27.071091 28154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:32:27.100212 28154 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:37349
I20260811 00:32:27.100211 28227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:37349 every 8 connection(s)
I20260811 00:32:27.104238 28154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:32:27.110018 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 28154
I20260811 00:32:27.110509 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:32:27.111780 2528 external_mini_cluster.cc:1428] Setting key 569ae9d5d4e3e0ce9479aa514b9d4166
I20260811 00:32:27.115999 28228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:32:27.128044 28228 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:32:27 dist-test-slave-fd7l krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408347, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.120.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.61@KRBTEST.COM" created.
I20260811 00:32:27.153115 28232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.133977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53818 (local address 127.2.120.62:37349)
0811 00:32:27.134789 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:27.134802 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:27.134856 (+ 54us) server_negotiation.cc:407] Connection header received
0811 00:32:27.135910 (+ 1054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:27.135916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:27.136159 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:27.136383 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:27.137743 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:27.138939 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:27.140334 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:27.140975 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:27.141201 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:27.144783 (+ 3582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:27.144792 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:27.144806 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:27.144835 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:27.147965 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:27.148876 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:27.148884 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:27.148887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:27.149116 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:27.149659 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:27.149664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:27.149667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:27.150483 (+ 816us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:27.150813 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:27.151470 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 00:32:27.151775 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":377,"threads_started":1}
I20260811 00:32:27.153159 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.132234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37349 (local address 127.0.0.1:53818)
0811 00:32:27.133220 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.133262 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 00:32:27.135496 (+ 2234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:32:27.136389 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:32:27.136396 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:32:27.137169 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:32:27.137519 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:27.137537 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:27.139104 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:27.139111 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:27.140188 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:27.140201 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:27.141216 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:27.141223 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:27.141436 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:27.142160 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:32:27.142185 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:32:27.144453 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:32:27.148119 (+ 3666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:27.148133 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:27.148149 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:27.148690 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:27.149238 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:27.149242 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:27.149244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:27.149405 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:27.150729 (+ 1324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:32:27.150736 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:32:27.150985 (+ 249us) client_negotiation.cc:769] Sending connection context
0811 00:32:27.151240 (+ 255us) client_negotiation.cc:240] Negotiation successful
0811 00:32:27.152009 (+ 769us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":162,"thread_start_us":484,"threads_started":1}
I20260811 00:32:27.156894 28228 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:32:27.158196 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.157668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:33993 (local address 127.0.0.1:47900)
0811 00:32:27.157881 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.158029 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 00:32:27.158571 28175 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.120.61:33993: connect: Connection refused (error 111)
I20260811 00:32:27.160375 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.159887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:33993 (local address 127.0.0.1:47906)
0811 00:32:27.160101 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.160195 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 00:32:27.160663 28228 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.61:33993: Network error: Client connection negotiation failed: client connection to 127.2.120.61:33993: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.61.keytab.
Entry for principal kudu/127.2.120.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.61.keytab.
I20260811 00:32:27.192920 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.61:33993
--webserver_interface=127.2.120.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.61.keytab
--principal=kudu/127.2.120.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:40597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:37349,127.2.120.61:33993,127.2.120.60:37335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:32:27.210042 28228 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } attempt: 1
I20260811 00:32:27.211417 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.210854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:33993 (local address 127.0.0.1:47918)
0811 00:32:27.211118 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.211249 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 00:32:27.212714 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.212213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:33993 (local address 127.0.0.1:47924)
0811 00:32:27.212398 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.212497 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 00:32:27.212950 28228 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.61:33993: Network error: Client connection negotiation failed: client connection to 127.2.120.61:33993: connect: Connection refused (error 111)
I20260811 00:32:27.281265 28228 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } attempt: 2
I20260811 00:32:27.282620 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.282096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:33993 (local address 127.0.0.1:47938)
0811 00:32:27.282381 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.282473 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 00:32:27.283864 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.283420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:33993 (local address 127.0.0.1:47948)
0811 00:32:27.283622 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.283723 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 00:32:27.284080 28228 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.61:33993: Network error: Client connection negotiation failed: client connection to 127.2.120.61:33993: connect: Connection refused (error 111)
I20260811 00:32:27.375554 28228 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } attempt: 3
I20260811 00:32:27.376784 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.376257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:33993 (local address 127.0.0.1:47960)
0811 00:32:27.376474 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.376634 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 00:32:27.377821 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.377462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:33993 (local address 127.0.0.1:47972)
0811 00:32:27.377625 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.377690 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 00:32:27.378003 28228 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.61:33993: Network error: Client connection negotiation failed: client connection to 127.2.120.61:33993: connect: Connection refused (error 111)
I20260811 00:32:27.521298 28228 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } attempt: 4
I20260811 00:32:27.522609 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.522024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:33993 (local address 127.0.0.1:47978)
0811 00:32:27.522364 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.522455 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260811 00:32:27.523769 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.523389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:33993 (local address 127.0.0.1:47990)
0811 00:32:27.523565 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.523649 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:33993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 00:32:27.523968 28228 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.61:33993: Network error: Client connection negotiation failed: client connection to 127.2.120.61:33993: connect: Connection refused (error 111)
W20260811 00:32:27.557425 28235 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 00:32:27.557868 28235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:32:27.558009 28235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:32:27.558159 28235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:32:27.565780 28235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:32:27.565951 28235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:32:27.566077 28235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:32:27.566172 28235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:32:27.566267 28235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:32:27.576565 28235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:40597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:37349,127.2.120.61:33993,127.2.120.60:37335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.61.keytab
--principal=kudu/127.2.120.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.61:33993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.120.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:32:27.578083 28235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:32:27.579638 28235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:32:27.599258 28244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:27.599350 28241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:27.599427 28246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:32:27.601255 28235 server_base.cc:1034] running on GCE node
I20260811 00:32:27.603448 28235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:32:27.607959 28235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:32:27.611706 28235 hybrid_clock.cc:648] HybridClock initialized: now 1786408347611576 us; error 157 us; skew 500 ppm
I20260811 00:32:27.612246 28235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:32:27 dist-test-slave-fd7l krb5kdc[28140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408347, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:32:27.616321 28235 init.cc:377] Logged in from keytab as kudu/127.2.120.61@KRBTEST.COM (short username kudu)
I20260811 00:32:27.618314 28235 webserver.cc:460] Webserver started at http://127.2.120.61:35935/ using document root <none> and password file <none>
I20260811 00:32:27.619187 28235 fs_manager.cc:359] Metadata directory not provided
I20260811 00:32:27.619357 28235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:32:27.619781 28235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:32:27.623577 28235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-1/data/instance:
uuid: "dc6820b5cb60423a9bd8b880103d9714"
format_stamp: "Formatted at 2026-08-11 00:32:27 on dist-test-slave-fd7l"
server_key: "9de40b603d88f03bfe7c5502310f42b6"
server_key_iv: "c1d588c3e6895045235a53dd3230409a"
server_key_version: "encryptionkey@0"
I20260811 00:32:27.624790 28235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-1/wal/instance:
uuid: "dc6820b5cb60423a9bd8b880103d9714"
format_stamp: "Formatted at 2026-08-11 00:32:27 on dist-test-slave-fd7l"
server_key: "9de40b603d88f03bfe7c5502310f42b6"
server_key_iv: "c1d588c3e6895045235a53dd3230409a"
server_key_version: "encryptionkey@0"
I20260811 00:32:27.631636 28235 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 00:32:27.636828 28254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:32:27.638589 28235 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:32:27.638845 28235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-1/wal
uuid: "dc6820b5cb60423a9bd8b880103d9714"
format_stamp: "Formatted at 2026-08-11 00:32:27 on dist-test-slave-fd7l"
server_key: "9de40b603d88f03bfe7c5502310f42b6"
server_key_iv: "c1d588c3e6895045235a53dd3230409a"
server_key_version: "encryptionkey@0"
I20260811 00:32:27.639147 28235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:32:27.668576 28235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:32:27.673774 28235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:32:27.674104 28235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:32:27.694814 28235 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.61:33993
I20260811 00:32:27.694835 28310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.61:33993 every 8 connection(s)
I20260811 00:32:27.696329 28235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-1/data/info.pb
I20260811 00:32:27.699928 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 28235
I20260811 00:32:27.700176 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-1/wal/instance
I20260811 00:32:27.700824 2528 external_mini_cluster.cc:1428] Setting key b7ce214a17a2da11d4567f281b25689c
I20260811 00:32:27.701275 28311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:32:27.713065 28311 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:32:27 dist-test-slave-fd7l krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408347, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.61@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.120.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.60@KRBTEST.COM" created.
I20260811 00:32:27.739233 28314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.717467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53826 (local address 127.2.120.62:37349)
0811 00:32:27.718084 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:27.718089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:27.719115 (+ 1026us) server_negotiation.cc:407] Connection header received
0811 00:32:27.721382 (+ 2267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:27.721388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:27.721499 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:27.721633 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:27.723469 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:27.724271 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:27.726055 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:27.726493 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:27.726648 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:27.730794 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:27.730800 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:27.730803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:27.730848 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:27.734709 (+ 3861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:27.736013 (+ 1304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:27.736020 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:27.736023 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:27.736193 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:27.736865 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:27.736878 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:27.736883 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:27.737262 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:27.737482 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:27.738416 (+ 934us) server_negotiation.cc:299] Negotiation successful
0811 00:32:27.738724 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":329,"threads_started":1}
I20260811 00:32:27.741459 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.718021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37349 (local address 127.0.0.1:53826)
0811 00:32:27.718924 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.718974 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 00:32:27.721041 (+ 2067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:32:27.721915 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:32:27.721926 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:32:27.722565 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:32:27.723214 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:27.723235 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:27.724472 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:27.724476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:27.725873 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:27.725889 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:27.726764 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:27.726771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:27.727031 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:27.728019 (+ 988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:32:27.728045 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:32:27.730495 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:32:27.734933 (+ 4438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:27.734945 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:27.734975 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:27.735807 (+ 832us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:27.736364 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:27.736376 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:27.736381 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:27.736701 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:27.737448 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:32:27.737460 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:32:27.737924 (+ 464us) client_negotiation.cc:769] Sending connection context
0811 00:32:27.738275 (+ 351us) client_negotiation.cc:240] Negotiation successful
0811 00:32:27.738952 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":406,"threads_started":1}
I20260811 00:32:27.743799 28311 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:32:27 dist-test-slave-fd7l krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408347, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.61@KRBTEST.COM for kudu/127.2.120.61@KRBTEST.COM
I20260811 00:32:27.762262 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.744785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:33993 (local address 127.0.0.1:48006)
0811 00:32:27.745002 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.745025 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:32:27.745157 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:32:27.746361 (+ 1204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:32:27.746365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:32:27.746861 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:32:27.746996 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:27.747005 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:27.748474 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:27.748482 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:27.749420 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:27.749440 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:27.750447 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:27.750454 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:27.750630 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:27.751303 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:32:27.751322 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:32:27.753657 (+ 2335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:32:27.757869 (+ 4212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:27.757878 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:27.757885 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:27.758477 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:27.759059 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:27.759066 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:27.759073 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:27.759293 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:27.760623 (+ 1330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:32:27.760632 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:32:27.760909 (+ 277us) client_negotiation.cc:769] Sending connection context
0811 00:32:27.761113 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 00:32:27.761528 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 00:32:27.762310 28317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.745139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48006 (local address 127.2.120.61:33993)
0811 00:32:27.745895 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:27.745901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:27.745929 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:32:27.745979 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:27.745986 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:27.746203 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:27.746372 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:27.747195 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:27.748330 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:27.749602 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:27.750260 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:27.750406 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:27.753924 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:27.753960 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:27.753985 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:27.754140 (+ 155us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:27.757628 (+ 3488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:27.758712 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:27.758721 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:27.758724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:27.758901 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:27.759587 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:27.759594 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:27.759597 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:27.760348 (+ 751us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:27.760707 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:27.761433 (+ 726us) server_negotiation.cc:299] Negotiation successful
0811 00:32:27.761771 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":449,"threads_started":1}
I20260811 00:32:27.766824 28311 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:32:27.768692 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.767845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46808)
0811 00:32:27.768126 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.768334 (+ 208us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260811 00:32:27.769161 28256 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
I20260811 00:32:27.771257 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.770685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46816)
0811 00:32:27.770924 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.771062 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 00:32:27.771600 28311 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:37335: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.60.keytab.
Entry for principal kudu/127.2.120.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.60.keytab.
I20260811 00:32:27.785408 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.60:37335
--webserver_interface=127.2.120.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.60.keytab
--principal=kudu/127.2.120.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:40597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:37349,127.2.120.61:33993,127.2.120.60:37335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:32:27.821043 28311 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } attempt: 1
I20260811 00:32:27.822809 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.822056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46822)
0811 00:32:27.822400 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.822578 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260811 00:32:27.823271 28228 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } attempt: 5
I20260811 00:32:27.824340 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.823890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46834)
0811 00:32:27.824062 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.824188 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 00:32:27.824707 28311 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:37335: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Aug 11 00:32:27 dist-test-slave-fd7l krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408347, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for kudu/127.2.120.61@KRBTEST.COM
I20260811 00:32:27.839121 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.824129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:33993 (local address 127.0.0.1:48008)
0811 00:32:27.824327 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.824347 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:32:27.824479 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:32:27.825107 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:32:27.825111 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:32:27.825507 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:32:27.825693 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:27.825702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:27.827119 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:27.827124 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:27.828055 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:27.828068 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:27.829212 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:27.829223 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:27.829640 (+ 417us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:27.830390 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:32:27.830409 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:32:27.832613 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:32:27.836368 (+ 3755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:27.836373 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:27.836397 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:27.836959 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:27.837414 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:27.837417 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:27.837420 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:27.837568 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:27.838234 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:32:27.838238 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:32:27.838430 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 00:32:27.838570 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 00:32:27.838830 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 00:32:27.839262 28317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.824450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48008 (local address 127.2.120.61:33993)
0811 00:32:27.824700 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:27.824707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:27.824728 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:32:27.824791 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:27.824797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:27.824904 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:27.825029 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:27.825844 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:27.826950 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:27.828204 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:27.828961 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:27.829149 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:27.832834 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:27.832841 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:27.832846 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:27.832888 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:27.836185 (+ 3297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:27.837095 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:27.837101 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:27.837104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:27.837268 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:27.837691 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:27.837696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:27.837699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:27.838080 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:27.838234 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:27.838718 (+ 484us) server_negotiation.cc:299] Negotiation successful
0811 00:32:27.838942 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260811 00:32:27.840497 28228 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:32:27.841799 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.841363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46840)
0811 00:32:27.841549 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.841647 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 00:32:27.842942 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.842559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46856)
0811 00:32:27.842713 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.842827 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 00:32:27.843194 28228 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:37335: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
I20260811 00:32:27.893015 28311 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } attempt: 2
I20260811 00:32:27.894462 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.893862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46872)
0811 00:32:27.894135 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.894253 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 00:32:27.894485 28228 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } attempt: 1
I20260811 00:32:27.895754 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.895197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46878)
0811 00:32:27.895421 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.895537 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 00:32:27.895772 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.895351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46886)
0811 00:32:27.895540 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.895632 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 00:32:27.896008 28311 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:37335: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
I20260811 00:32:27.896881 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.896477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46896)
0811 00:32:27.896686 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.896770 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260811 00:32:27.897143 28228 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:37335: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
I20260811 00:32:27.965431 28228 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } attempt: 2
I20260811 00:32:27.966984 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.966218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46898)
0811 00:32:27.966620 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.966768 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183}
I20260811 00:32:27.968408 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.967906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46900)
0811 00:32:27.968120 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.968236 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 00:32:27.968727 28228 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:37335: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
I20260811 00:32:27.987322 28311 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } attempt: 3
I20260811 00:32:27.988763 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.988082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46912)
0811 00:32:27.988363 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.988494 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 00:32:27.990579 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:27.989833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46918)
0811 00:32:27.990241 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:27.990373 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248}
W20260811 00:32:27.990923 28311 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:37335: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
I20260811 00:32:28.075018 28228 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } attempt: 3
I20260811 00:32:28.076454 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.075774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46930)
0811 00:32:28.076067 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:28.076202 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 00:32:28.077860 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.077349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46940)
0811 00:32:28.077599 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:28.077697 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260811 00:32:28.078322 28228 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:37335: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
I20260811 00:32:28.134181 28311 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } attempt: 4
I20260811 00:32:28.135524 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.134897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46946)
0811 00:32:28.135187 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:28.135313 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 00:32:28.136749 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.136286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46952)
0811 00:32:28.136483 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:28.136598 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 00:32:28.137023 28311 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:37335: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
W20260811 00:32:28.195279 28320 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 00:32:28.195629 28320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:32:28.195699 28320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:32:28.195784 28320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:32:28.203523 28320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:32:28.203655 28320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:32:28.203716 28320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:32:28.203761 28320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:32:28.203826 28320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:32:28.214174 28320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:40597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:37349,127.2.120.61:33993,127.2.120.60:37335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.60.keytab
--principal=kudu/127.2.120.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.60:37335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.120.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:32:28.215600 28320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:32:28.217073 28320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:32:28.230211 28329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:28.230139 28327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:28.230217 28326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:32:28.231134 28320 server_base.cc:1034] running on GCE node
I20260811 00:32:28.231974 28320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:32:28.233511 28320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:32:28.234762 28320 hybrid_clock.cc:648] HybridClock initialized: now 1786408348234658 us; error 56 us; skew 500 ppm
I20260811 00:32:28.235177 28320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:32:28 dist-test-slave-fd7l krb5kdc[28140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408348, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:32:28.243701 28320 init.cc:377] Logged in from keytab as kudu/127.2.120.60@KRBTEST.COM (short username kudu)
I20260811 00:32:28.245570 28320 webserver.cc:460] Webserver started at http://127.2.120.60:40307/ using document root <none> and password file <none>
I20260811 00:32:28.246345 28320 fs_manager.cc:359] Metadata directory not provided
I20260811 00:32:28.246498 28320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:32:28.246842 28320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:32:28.250190 28320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-2/data/instance:
uuid: "eb54935d7cc34160a14ed339b0efa0bf"
format_stamp: "Formatted at 2026-08-11 00:32:28 on dist-test-slave-fd7l"
server_key: "b976bcf8475dd7622f1eaa8f60afe4c1"
server_key_iv: "320e1cfe14c18b7f58e5ec38482984f4"
server_key_version: "encryptionkey@0"
I20260811 00:32:28.251012 28320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-2/wal/instance:
uuid: "eb54935d7cc34160a14ed339b0efa0bf"
format_stamp: "Formatted at 2026-08-11 00:32:28 on dist-test-slave-fd7l"
server_key: "b976bcf8475dd7622f1eaa8f60afe4c1"
server_key_iv: "320e1cfe14c18b7f58e5ec38482984f4"
server_key_version: "encryptionkey@0"
I20260811 00:32:28.255604 28228 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } attempt: 4
I20260811 00:32:28.256940 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.256347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46958)
0811 00:32:28.256663 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:28.256768 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 00:32:28.257504 28320 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:32:28.258170 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.257727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46968)
0811 00:32:28.257893 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:28.258001 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 00:32:28.258402 28228 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:37335: Network error: Client connection negotiation failed: client connection to 127.2.120.60:37335: connect: Connection refused (error 111)
I20260811 00:32:28.261950 28340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:32:28.263733 28320 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260811 00:32:28.263921 28320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-2/wal
uuid: "eb54935d7cc34160a14ed339b0efa0bf"
format_stamp: "Formatted at 2026-08-11 00:32:28 on dist-test-slave-fd7l"
server_key: "b976bcf8475dd7622f1eaa8f60afe4c1"
server_key_iv: "320e1cfe14c18b7f58e5ec38482984f4"
server_key_version: "encryptionkey@0"
I20260811 00:32:28.264155 28320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:32:28.290134 28320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:32:28.294770 28320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:32:28.295025 28320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:32:28.315910 28320 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.60:37335
I20260811 00:32:28.315936 28399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.60:37335 every 8 connection(s)
I20260811 00:32:28.317301 28320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-2/data/info.pb
I20260811 00:32:28.321409 28400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:32:28.327345 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 28320
I20260811 00:32:28.327553 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407939260739-2528-0/minicluster-data/master-2/wal/instance
I20260811 00:32:28.328138 2528 external_mini_cluster.cc:1428] Setting key 935c96d26d77fd48053480a54a85ceeb
I20260811 00:32:28.329150 28400 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:32:28 dist-test-slave-fd7l krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408348, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.60@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
Aug 11 00:32:28 dist-test-slave-fd7l krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.60@KRBTEST.COM
I20260811 00:32:28.349774 28404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.332225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53832 (local address 127.2.120.62:37349)
0811 00:32:28.332865 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:28.332874 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:28.333512 (+ 638us) server_negotiation.cc:407] Connection header received
0811 00:32:28.334185 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:28.334192 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:28.334297 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:28.334421 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:28.335758 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.336686 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.338416 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.339127 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.339228 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:28.342844 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:28.342852 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:28.342857 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:28.342905 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:28.346116 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.347177 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.347181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.347184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.347316 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.347800 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.347803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.347805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.348090 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:28.348266 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:28.349103 (+ 837us) server_negotiation.cc:299] Negotiation successful
0811 00:32:28.349335 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":340,"threads_started":1}
I20260811 00:32:28.350773 28405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.332246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37349 (local address 127.0.0.1:53832)
0811 00:32:28.333318 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:28.333363 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 00:32:28.333899 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:32:28.334519 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:32:28.334526 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:32:28.335226 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:32:28.335577 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:28.335593 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.336850 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.336857 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:28.338254 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:28.338266 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.339350 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.339357 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:28.339572 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:28.340392 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:32:28.340415 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:32:28.342588 (+ 2173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:32:28.346256 (+ 3668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:28.346267 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:28.346279 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:28.347037 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:28.347441 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:28.347445 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:28.347448 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:28.347682 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:28.348241 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:32:28.348249 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:32:28.348706 (+ 457us) client_negotiation.cc:769] Sending connection context
0811 00:32:28.348998 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 00:32:28.349498 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":857,"thread_start_us":699,"threads_started":1}
I20260811 00:32:28.351531 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.330328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46980 (local address 127.2.120.60:37335)
0811 00:32:28.331151 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:28.331161 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:28.331204 (+ 43us) server_negotiation.cc:407] Connection header received
0811 00:32:28.333885 (+ 2681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:28.333905 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:28.334160 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:28.334312 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:28.335938 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.337433 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.338975 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.339683 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.339824 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:28.344706 (+ 4882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:28.344714 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:28.344727 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:28.344757 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:28.348225 (+ 3468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.349079 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.349088 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.349091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.349261 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.349703 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.349708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.349712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.350172 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:28.350514 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:28.350832 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 00:32:28.351096 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":350,"threads_started":1}
I20260811 00:32:28.352631 28400 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:32:28 dist-test-slave-fd7l krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408348, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.60@KRBTEST.COM for kudu/127.2.120.61@KRBTEST.COM
Aug 11 00:32:28 dist-test-slave-fd7l krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:32:28.367777 28405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.353430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:33993 (local address 127.0.0.1:48012)
0811 00:32:28.353638 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:28.353661 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:32:28.353790 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:32:28.354459 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:32:28.354464 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:32:28.354946 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:32:28.355194 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:28.355210 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.356353 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.356360 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:28.357483 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:28.357496 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.358197 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.358203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:28.358386 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:28.358987 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:32:28.359011 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:32:28.361262 (+ 2251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:32:28.364675 (+ 3413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:28.364682 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:28.364689 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:28.365231 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:28.365716 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:28.365724 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:28.365729 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:28.365942 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:28.366698 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:32:28.366704 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:32:28.366888 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 00:32:28.367075 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 00:32:28.367384 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 00:32:28.367796 28406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.353575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48012 (local address 127.2.120.61:33993)
0811 00:32:28.354143 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:28.354149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:28.354167 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:32:28.354230 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:28.354234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:28.354292 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:28.354375 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:28.355392 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.356191 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.357654 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.358042 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.358148 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:28.361478 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:28.361487 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:28.361493 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:28.361539 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:28.364487 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.365398 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.365402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.365405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.365537 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.366102 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.366113 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.366117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.366466 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:28.366597 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:28.367224 (+ 627us) server_negotiation.cc:299] Negotiation successful
0811 00:32:28.367403 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":321,"threads_started":1}
I20260811 00:32:28.369081 28400 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:32:28.371455 28404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.354907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53834 (local address 127.2.120.62:37349)
0811 00:32:28.355078 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:28.355082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:28.355097 (+ 15us) server_negotiation.cc:407] Connection header received
0811 00:32:28.355216 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:28.355220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:28.355312 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:28.355422 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:28.356806 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.357452 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.358988 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.359382 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.359473 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:28.364177 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:28.364182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:28.364185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:28.364217 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:28.368348 (+ 4131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.369266 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.369272 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.369276 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.369417 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.369834 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.369838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.369840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.370210 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:28.370476 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:28.370862 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 00:32:28.371106 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 00:32:28 dist-test-slave-fd7l krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408348, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.60@KRBTEST.COM for kudu/127.2.120.60@KRBTEST.COM
Aug 11 00:32:28 dist-test-slave-fd7l krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.61@KRBTEST.COM
I20260811 00:32:28.384065 28405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.369910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46988)
0811 00:32:28.370099 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:28.370117 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:32:28.370219 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:32:28.370577 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:32:28.370582 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:32:28.371092 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:32:28.371263 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:28.371276 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.372572 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.372580 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:28.373740 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:28.373753 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.374472 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.374480 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:28.374715 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:28.375407 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:32:28.375428 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:32:28.377650 (+ 2222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:32:28.381459 (+ 3809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:28.381463 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:28.381466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:28.382010 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:28.382457 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:28.382465 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:28.382479 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:28.382656 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:28.383212 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:32:28.383218 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:32:28.383391 (+ 173us) client_negotiation.cc:769] Sending connection context
0811 00:32:28.383557 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 00:32:28.383785 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 00:32:28.384387 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.370065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46988 (local address 127.2.120.60:37335)
0811 00:32:28.370217 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:28.370221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:28.370236 (+ 15us) server_negotiation.cc:407] Connection header received
0811 00:32:28.370356 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:28.370361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:28.370430 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:28.370536 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:28.371408 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.372409 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.373885 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.374310 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.374407 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:28.377834 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:28.377839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:28.377842 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:28.377871 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:28.381297 (+ 3426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.382156 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.382160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.382162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.382289 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.382779 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.382782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.382784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.383077 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:28.383192 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:28.383704 (+ 512us) server_negotiation.cc:299] Negotiation successful
0811 00:32:28.383962 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 00:32:28.390413 28406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.373276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48014 (local address 127.2.120.61:33993)
0811 00:32:28.373421 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:28.373425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:28.373441 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:32:28.373568 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:28.373572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:28.373630 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:28.373704 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:28.375259 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.376006 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.377549 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.378043 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.378140 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:28.383489 (+ 5349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:28.383494 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:28.383498 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:28.383555 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:28.387227 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.387999 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.388005 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.388008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.388172 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.388699 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.388705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.388715 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.389208 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:28.389542 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:28.389841 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 00:32:28.390086 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 00:32:28.390405 28400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf: Bootstrap starting.
I20260811 00:32:28.394685 28400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf: Neither blocks nor log segments found. Creating new log.
I20260811 00:32:28.395479 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:32:28.395939 28400 log.cc:824] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf: Log is configured to *not* fsync() on all Append() calls
I20260811 00:32:28.399166 28400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf: No bootstrap required, opened a new log
I20260811 00:32:28.398677 28201 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c461f11505e0407cacf14cd7a6a51a7b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53834
I20260811 00:32:28.408912 28400 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } } peers { permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } } peers { permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } }
I20260811 00:32:28.409281 28400 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb54935d7cc34160a14ed339b0efa0bf, State: Initialized, Role: FOLLOWER
I20260811 00:32:28.409772 28400 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } } peers { permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } } peers { permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } }
I20260811 00:32:28.410794 28408 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } } peers { permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } } peers { permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } } }
I20260811 00:32:28.411196 28408 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:32:28.411813 28400 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf [sys.catalog]: configured and running, proceeding with master startup.
W20260811 00:32:28.423029 28420 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:32:28.423225 28420 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:32:28.420565 28320 master_runner.cc:186] Error getting master registration for 127.2.120.62:37349: OK, instance_id { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" instance_seqno: 1786408347088829 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 00:32:28.432693 28320 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 00:32:28.436244 28311 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } attempt: 5
Aug 11 00:32:28 dist-test-slave-fd7l krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408347, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.61@KRBTEST.COM for kudu/127.2.120.60@KRBTEST.COM
I20260811 00:32:28.454430 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.437469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46994 (local address 127.2.120.60:37335)
0811 00:32:28.437690 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:28.437696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:28.437715 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:32:28.437802 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:28.437806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:28.437882 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:28.438010 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:28.438906 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.439728 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.441058 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.441590 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.441749 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:28.446093 (+ 4344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:28.446099 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:28.446102 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:28.446137 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:28.450222 (+ 4085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.451551 (+ 1329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.451558 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.451562 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.451776 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.452488 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.452493 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.452496 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.453033 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:28.453220 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:28.453716 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 00:32:28.454033 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 00:32:28.454463 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.437112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:46994)
0811 00:32:28.437328 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:28.437353 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:32:28.437496 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:32:28.438043 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:32:28.438049 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:32:28.438552 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:32:28.438726 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:28.438735 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.439913 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.439920 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:28.440896 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:28.440909 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.441760 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.441767 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:28.442063 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:28.442787 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:32:28.442815 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:32:28.445845 (+ 3030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:32:28.450620 (+ 4775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:28.450630 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:28.450635 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:28.451356 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:28.452027 (+ 671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:28.452035 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:28.452040 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:28.452318 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:28.453180 (+ 862us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:32:28.453186 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:32:28.453395 (+ 209us) client_negotiation.cc:769] Sending connection context
0811 00:32:28.453574 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 00:32:28.453970 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 00:32:28.461110 28311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714: Bootstrap starting.
I20260811 00:32:28.465814 28311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714: Neither blocks nor log segments found. Creating new log.
I20260811 00:32:28.467275 28311 log.cc:824] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714: Log is configured to *not* fsync() on all Append() calls
I20260811 00:32:28.470978 28311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714: No bootstrap required, opened a new log
I20260811 00:32:28.482926 28311 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } } peers { permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } } peers { permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } }
I20260811 00:32:28.483441 28311 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc6820b5cb60423a9bd8b880103d9714, State: Initialized, Role: FOLLOWER
I20260811 00:32:28.484073 28311 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } } peers { permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } } peers { permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } }
I20260811 00:32:28.485249 28424 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } } peers { permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } } peers { permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } } }
I20260811 00:32:28.485757 28424 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:32:28.486735 28311 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 00:32:28.505239 28437 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:32:28.505553 28437 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:32:28.501365 28235 master_runner.cc:186] Error getting master registration for 127.2.120.62:37349: OK, instance_id { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" instance_seqno: 1786408347088829 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 00:32:28.516109 28235 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 00:32:28.535691 28228 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } attempt: 5
Aug 11 00:32:28 dist-test-slave-fd7l krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408347, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for kudu/127.2.120.60@KRBTEST.COM
W20260811 00:32:28.547590 28346 tablet.cc:2369] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:32:28.558892 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.536606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:37335 (local address 127.0.0.1:47000)
0811 00:32:28.536923 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:28.536944 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:32:28.537096 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:32:28.537573 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:32:28.537578 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:32:28.538179 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:32:28.538406 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:28.538420 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.540259 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.540279 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:28.542257 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:28.542281 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.543626 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.543635 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:28.543915 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:28.544808 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:32:28.544838 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:32:28.548977 (+ 4139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:32:28.555143 (+ 6166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:28.555151 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:28.555156 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:28.555877 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:28.556564 (+ 687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:28.556570 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:28.556576 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:28.556818 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:28.557693 (+ 875us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:32:28.557699 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:32:28.557884 (+ 185us) client_negotiation.cc:769] Sending connection context
0811 00:32:28.558056 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 00:32:28.558418 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260811 00:32:28.560104 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.536739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47000 (local address 127.2.120.60:37335)
0811 00:32:28.536981 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:28.536988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:28.537020 (+ 32us) server_negotiation.cc:407] Connection header received
0811 00:32:28.537279 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:28.537286 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:28.537378 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:28.537500 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:28.538590 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.539958 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.542565 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.543368 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.543534 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:28.549363 (+ 5829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:28.549374 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:28.549380 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:28.549438 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:28.554816 (+ 5378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.556130 (+ 1314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.556138 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.556142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.556370 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.556974 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.556981 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.557000 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.557431 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:28.557608 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:28.559372 (+ 1764us) server_negotiation.cc:299] Negotiation successful
0811 00:32:28.559701 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 00:32:28.567807 28228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b: Bootstrap starting.
I20260811 00:32:28.573407 28228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b: Neither blocks nor log segments found. Creating new log.
I20260811 00:32:28.575207 28228 log.cc:824] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b: Log is configured to *not* fsync() on all Append() calls
I20260811 00:32:28.585738 28228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b: No bootstrap required, opened a new log
I20260811 00:32:28.594645 28228 raft_consensus.cc:348] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } } peers { permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } } peers { permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } }
I20260811 00:32:28.595283 28228 raft_consensus.cc:729] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c461f11505e0407cacf14cd7a6a51a7b, State: Initialized, Role: FOLLOWER
I20260811 00:32:28.595928 28228 consensus_queue.cc:260] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } } peers { permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } } peers { permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } }
I20260811 00:32:28.597659 28440 sys_catalog.cc:455] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } } peers { permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } } peers { permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } } }
I20260811 00:32:28.598333 28440 sys_catalog.cc:458] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:32:28.599154 28228 sys_catalog.cc:564] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:32:28.599226 28201 raft_consensus.cc:482] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 00:32:28.599466 28201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:32:28.603219 28201 raft_consensus.cc:504] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } } peers { permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } } peers { permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } }
I20260811 00:32:28.606161 28283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c461f11505e0407cacf14cd7a6a51a7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dc6820b5cb60423a9bd8b880103d9714"
I20260811 00:32:28.607159 28283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:32:28.610972 28283 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c461f11505e0407cacf14cd7a6a51a7b in term 1.
I20260811 00:32:28.611864 28175 leader_election.cc:302] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c461f11505e0407cacf14cd7a6a51a7b, dc6820b5cb60423a9bd8b880103d9714; no voters:
I20260811 00:32:28.616961 28440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:32:28.617717 28369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c461f11505e0407cacf14cd7a6a51a7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "eb54935d7cc34160a14ed339b0efa0bf"
I20260811 00:32:28.618279 28369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:32:28.622053 28369 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c461f11505e0407cacf14cd7a6a51a7b in term 1.
I20260811 00:32:28.622869 28201 leader_election.cc:288] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [CANDIDATE]: Term 1 election: Requested vote from peers dc6820b5cb60423a9bd8b880103d9714 (127.2.120.61:33993), eb54935d7cc34160a14ed339b0efa0bf (127.2.120.60:37335)
I20260811 00:32:28.630316 28440 raft_consensus.cc:686] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [term 1 LEADER]: Becoming Leader. State: Replica: c461f11505e0407cacf14cd7a6a51a7b, State: Running, Role: LEADER
I20260811 00:32:28.631428 28440 consensus_queue.cc:237] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } } peers { permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } } peers { permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } }
I20260811 00:32:28.643826 28445 sys_catalog.cc:455] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c461f11505e0407cacf14cd7a6a51a7b. Latest consensus state: current_term: 1 leader_uuid: "c461f11505e0407cacf14cd7a6a51a7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } } peers { permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } } peers { permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } } }
I20260811 00:32:28.644255 28445 sys_catalog.cc:458] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:28.649268 28450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:32:28.669157 28450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260811 00:32:28.677423 28260 tablet.cc:2369] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:32:28.721657 28369 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf [term 1 FOLLOWER]: Refusing update from remote peer c461f11505e0407cacf14cd7a6a51a7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:32:28.721657 28283 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714 [term 1 FOLLOWER]: Refusing update from remote peer c461f11505e0407cacf14cd7a6a51a7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:32:28.724908 28440 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 00:32:28.733660 28445 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 00:32:28.762362 28471 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 00:32:28.769539 28424 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c461f11505e0407cacf14cd7a6a51a7b. Latest consensus state: current_term: 1 leader_uuid: "c461f11505e0407cacf14cd7a6a51a7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } } peers { permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } } peers { permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } } }
I20260811 00:32:28.769960 28424 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:32:28.780351 28461 mvcc.cc:205] Tried to move back new op lower bound from 7317128596292276224 to 7317128596010893312. Current Snapshot: MvccSnapshot[applied={T|T < 7317128596292276224}]
I20260811 00:32:28.781076 28424 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c461f11505e0407cacf14cd7a6a51a7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } } peers { permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } } peers { permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } } }
I20260811 00:32:28.781410 28424 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:32:28.792634 28445 sys_catalog.cc:455] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c461f11505e0407cacf14cd7a6a51a7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } } peers { permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } } peers { permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } } }
I20260811 00:32:28.792971 28445 sys_catalog.cc:458] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:28.800704 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.663840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47012 (local address 127.2.120.60:37335)
0811 00:32:28.664034 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:28.664053 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:28.681104 (+ 17051us) server_negotiation.cc:407] Connection header received
0811 00:32:28.684217 (+ 3113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:28.684224 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:28.684328 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:28.684469 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:28.689705 (+ 5236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.690842 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.712932 (+ 22090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.714351 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.714750 (+ 399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:28.767848 (+ 53098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:28.767857 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:28.767863 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:28.767915 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:28.772019 (+ 4104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.795889 (+ 23870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.795897 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.795901 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.796132 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.798110 (+ 1978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.798116 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.798119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.798579 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:28.798856 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:28.799937 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0811 00:32:28.800265 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 00:32:28.805630 28406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.661620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48018 (local address 127.2.120.61:33993)
0811 00:32:28.661809 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:28.661818 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:28.679633 (+ 17815us) server_negotiation.cc:407] Connection header received
0811 00:32:28.681578 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:28.681587 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:28.681677 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:28.681753 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:28.688078 (+ 6325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.689283 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.711682 (+ 22399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.712393 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.712534 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:28.773214 (+ 60680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:28.773228 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:28.773234 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:28.773294 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:28.777667 (+ 4373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.796924 (+ 19257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.796930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.796934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.799450 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.800090 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.800095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.800098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.803249 (+ 3151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:28.803695 (+ 446us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:28.804953 (+ 1258us) server_negotiation.cc:299] Negotiation successful
0811 00:32:28.805216 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 00:32:28.806129 28472 mvcc.cc:205] Tried to move back new op lower bound from 7317128596292276224 to 7317128596010893312. Current Snapshot: MvccSnapshot[applied={T|T < 7317128596292276224 or (T in {7317128596292276224})}]
I20260811 00:32:28.806360 28408 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader c461f11505e0407cacf14cd7a6a51a7b. Latest consensus state: current_term: 1 leader_uuid: "c461f11505e0407cacf14cd7a6a51a7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } } peers { permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } } peers { permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } } }
I20260811 00:32:28.806704 28408 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:32:28.810497 28404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:28.676848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53846 (local address 127.2.120.62:37349)
0811 00:32:28.677079 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:28.677089 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:28.687094 (+ 10005us) server_negotiation.cc:407] Connection header received
0811 00:32:28.687390 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:28.687398 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:28.687486 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:28.687558 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:28.714251 (+ 26693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.716068 (+ 1817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.763902 (+ 47834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:28.764755 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:28.764884 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:28.798128 (+ 33244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:28.798136 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:28.798140 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:28.798223 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:28.802302 (+ 4079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.803291 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.803299 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.803305 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.803638 (+ 333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:28.804166 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:28.804172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:28.804175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:28.804648 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:28.806270 (+ 1622us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:28.806421 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 00:32:28.806672 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 00:32:28.813015 28450 catalog_manager.cc:1347] Generated new cluster ID: 7479bf6dc2044fe3a238bf9c96da095e
I20260811 00:32:28.813140 28450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:32:28.813735 28440 sys_catalog.cc:455] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c461f11505e0407cacf14cd7a6a51a7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } } peers { permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } } peers { permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } } }
I20260811 00:32:28.814100 28440 sys_catalog.cc:458] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:28.815243 28470 mvcc.cc:205] Tried to move back new op lower bound from 7317128596292276224 to 7317128596010893312. Current Snapshot: MvccSnapshot[applied={T|T < 7317128596292276224}]
I20260811 00:32:28.828579 28408 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c461f11505e0407cacf14cd7a6a51a7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c461f11505e0407cacf14cd7a6a51a7b" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37349 } } peers { permanent_uuid: "dc6820b5cb60423a9bd8b880103d9714" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 33993 } } peers { permanent_uuid: "eb54935d7cc34160a14ed339b0efa0bf" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 37335 } } }
I20260811 00:32:28.828924 28408 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:32:28.863458 28450 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:32:28.865540 28450 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:32:28.881502 28450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c461f11505e0407cacf14cd7a6a51a7b: Generated new TSK 0
I20260811 00:32:28.882649 28450 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 00:32:29.032801 28404 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:29.023079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53862 (local address 127.2.120.62:37349)
0811 00:32:29.023337 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:29.023344 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:29.030106 (+ 6762us) server_negotiation.cc:407] Connection header received
0811 00:32:29.030942 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:29.030950 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:29.031066 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:29.031153 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:29.032478 (+ 1325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53862: BlockingRecv error: recv got EOF from 127.0.0.1:53862 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 00:32:29.034925 28406 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:29.015522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48024 (local address 127.2.120.61:33993)
0811 00:32:29.015787 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:29.015795 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:29.021119 (+ 5324us) server_negotiation.cc:407] Connection header received
0811 00:32:29.032896 (+ 11777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:29.032903 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:29.033013 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:29.033101 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:29.034643 (+ 1542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48024: BlockingRecv error: recv got EOF from 127.0.0.1:48024 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 00:32:29.035017 28403 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:29.017805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47026 (local address 127.2.120.60:37335)
0811 00:32:29.018031 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:29.018039 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:29.029570 (+ 11531us) server_negotiation.cc:407] Connection header received
0811 00:32:29.033446 (+ 3876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:29.033453 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:29.033552 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:29.033640 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:29.034749 (+ 1109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47026: BlockingRecv error: recv got EOF from 127.0.0.1:47026 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 00:32:29.091517 28406 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:29.072100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48030 (local address 127.2.120.61:33993)
0811 00:32:29.072677 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:29.072692 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:29.082119 (+ 9427us) server_negotiation.cc:407] Connection header received
0811 00:32:29.082818 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:29.082829 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:29.083267 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:29.083530 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:29.091189 (+ 7659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48030: BlockingRecv error: recv got EOF from 127.0.0.1:48030 (error 108)
Metrics: {"server-negotiator.queue_time_us":241}
W20260811 00:32:29.152401 28406 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:29.146226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48046 (local address 127.2.120.61:33993)
0811 00:32:29.146421 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:29.146428 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:29.150259 (+ 3831us) server_negotiation.cc:407] Connection header received
0811 00:32:29.150780 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:29.150786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:29.150893 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:29.150979 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:29.152116 (+ 1137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48046: BlockingRecv error: recv got EOF from 127.0.0.1:48046 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 00:32:29.171022 28406 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:29.167897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48056 (local address 127.2.120.61:33993)
0811 00:32:29.168110 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:29.168116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:29.169053 (+ 937us) server_negotiation.cc:407] Connection header received
0811 00:32:29.169572 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:29.169578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:29.169719 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:29.169851 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:29.170768 (+ 917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48056: BlockingRecv error: recv got EOF from 127.0.0.1:48056 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260811 00:32:29.182448 28406 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:29.179588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48062 (local address 127.2.120.61:33993)
0811 00:32:29.179923 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:29.179931 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:29.180602 (+ 671us) server_negotiation.cc:407] Connection header received
0811 00:32:29.180882 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:29.180888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:29.180993 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:29.181097 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:29.182185 (+ 1088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48062: BlockingRecv error: recv got EOF from 127.0.0.1:48062 (error 108)
Metrics: {"server-negotiator.queue_time_us":135}
W20260811 00:32:29.197307 28406 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:29.194655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48066 (local address 127.2.120.61:33993)
0811 00:32:29.194869 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:29.194875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:29.195388 (+ 513us) server_negotiation.cc:407] Connection header received
0811 00:32:29.195688 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:29.195694 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:29.195832 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:29.195941 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:29.196994 (+ 1053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48066: BlockingRecv error: recv got EOF from 127.0.0.1:48066 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 00:32:29.221680 28406 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:29.219226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48076 (local address 127.2.120.61:33993)
0811 00:32:29.219494 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:29.219502 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:29.219980 (+ 478us) server_negotiation.cc:407] Connection header received
0811 00:32:29.220182 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:29.220189 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:29.220305 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:29.220459 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:29.221411 (+ 952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48076: BlockingRecv error: recv got EOF from 127.0.0.1:48076 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260811 00:32:29.260061 28406 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:29.257867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48088 (local address 127.2.120.61:33993)
0811 00:32:29.258094 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:29.258102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:29.258388 (+ 286us) server_negotiation.cc:407] Connection header received
0811 00:32:29.258630 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:29.258634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:29.258711 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:29.258806 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:29.259802 (+ 996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48088: BlockingRecv error: recv got EOF from 127.0.0.1:48088 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260811 00:32:29.332415 28406 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:29.329753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48090 (local address 127.2.120.61:33993)
0811 00:32:29.329967 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:29.329974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:29.330603 (+ 629us) server_negotiation.cc:407] Connection header received
0811 00:32:29.330852 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:29.330859 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:29.330988 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:29.331124 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:29.332133 (+ 1009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48090: BlockingRecv error: recv got EOF from 127.0.0.1:48090 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:32:29.425441 28420 catalog_manager.cc:1259] Loaded cluster ID: 7479bf6dc2044fe3a238bf9c96da095e
I20260811 00:32:29.425556 28420 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf: loading cluster ID for follower catalog manager: success
I20260811 00:32:29.428686 28420 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf: acquiring CA information for follower catalog manager: success
I20260811 00:32:29.430646 28420 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eb54935d7cc34160a14ed339b0efa0bf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 00:32:29.471225 28406 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:29.468576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48100 (local address 127.2.120.61:33993)
0811 00:32:29.468831 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:29.468838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:29.469448 (+ 610us) server_negotiation.cc:407] Connection header received
0811 00:32:29.469741 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:29.469750 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:29.469861 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:29.469984 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:29.470961 (+ 977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48100: BlockingRecv error: recv got EOF from 127.0.0.1:48100 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
I20260811 00:32:29.507781 28437 catalog_manager.cc:1259] Loaded cluster ID: 7479bf6dc2044fe3a238bf9c96da095e
I20260811 00:32:29.507890 28437 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714: loading cluster ID for follower catalog manager: success
I20260811 00:32:29.510557 28437 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714: acquiring CA information for follower catalog manager: success
I20260811 00:32:29.512651 28437 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dc6820b5cb60423a9bd8b880103d9714: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:32:29.737298 28406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:29.731661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48110 (local address 127.2.120.61:33993)
0811 00:32:29.731891 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:29.731898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:29.732128 (+ 230us) server_negotiation.cc:407] Connection header received
0811 00:32:29.732291 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:29.732296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:29.732466 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:29.732597 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:32:29.733048 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:29.733733 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:29.735032 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:29.735428 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:29.735525 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:29.735997 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:32:29.736557 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:32:29.736666 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:29.736874 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 00:32:29.736948 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 00:32:29.756008 28553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:29.748851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47042 (local address 127.2.120.60:37335)
0811 00:32:29.749652 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:29.749657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:29.749679 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:32:29.749785 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:29.749789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:29.749966 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:29.750147 (+ 181us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:32:29.750569 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:29.751372 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:29.752956 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:29.753638 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:29.753764 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:29.754532 (+ 768us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:32:29.755124 (+ 592us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:32:29.755260 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:29.755520 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 00:32:29.755666 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":573,"threads_started":1}
I20260811 00:32:29.770905 28553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:29.763362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47056 (local address 127.2.120.60:37335)
0811 00:32:29.763617 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:29.763627 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:29.764965 (+ 1338us) server_negotiation.cc:407] Connection header received
0811 00:32:29.765285 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:29.765294 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:29.765396 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:29.765512 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:32:29.766104 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:29.767105 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:29.769203 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:29.769603 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:29.769753 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:29.770169 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:32:29.770274 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:32:29.770377 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:29.770537 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 00:32:29.770620 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 00:32:29.771802 28406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:29.763662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48122 (local address 127.2.120.61:33993)
0811 00:32:29.763903 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:29.763909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:29.765170 (+ 1261us) server_negotiation.cc:407] Connection header received
0811 00:32:29.765727 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:29.765735 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:29.765845 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:29.765935 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:32:29.766517 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:29.767550 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:29.769826 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:29.770269 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:29.770414 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:29.770926 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:32:29.771116 (+ 190us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:32:29.771204 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:29.771393 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 00:32:29.771480 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 00:32:29.773988 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:29.763023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53866 (local address 127.2.120.62:37349)
0811 00:32:29.764158 (+ 1135us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:29.764167 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:29.765760 (+ 1593us) server_negotiation.cc:407] Connection header received
0811 00:32:29.766014 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:29.766020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:29.766288 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:29.766491 (+ 203us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:32:29.767041 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:29.768137 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:29.771682 (+ 3545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:29.772274 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:29.772411 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:29.772990 (+ 579us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:32:29.773381 (+ 391us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:32:29.773508 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:29.773629 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 00:32:29.773716 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":971,"thread_start_us":845,"threads_started":1}
I20260811 00:32:29.779237 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 28154
I20260811 00:32:29.780375 28222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:32:29.891891 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 28154
I20260811 00:32:29.920042 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 28235
I20260811 00:32:29.921425 28305 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:32:30.030807 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 28235
I20260811 00:32:30.055896 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 28320
I20260811 00:32:30.057116 28395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:32:30.167793 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 28320
2026-08-11T00:32:30Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5247 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:32:30 dist-test-slave-fd7l krb5kdc[28571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:32:30 dist-test-slave-fd7l krb5kdc[28571](info): set up 2 sockets
Aug 11 00:32:30 dist-test-slave-fd7l krb5kdc[28571](info): commencing operation
krb5kdc: starting...
W20260811 00:32:31.448223 2528 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.008s sys 0.061s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/test-user.keytab.
Aug 11 00:32:31 dist-test-slave-fd7l krb5kdc[28571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:32:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:32:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.120.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
Entry for principal kudu/127.2.120.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab.
I20260811 00:32:31.864789 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:37597
--webserver_interface=127.2.120.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:44735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:37597,127.2.120.61:44449,127.2.120.60:40595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:32:32.239310 28586 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 00:32:32.239773 28586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:32:32.239908 28586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:32:32.240032 28586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:32:32.248899 28586 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:32:32.249029 28586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:32:32.249150 28586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:32:32.249240 28586 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:32:32.249366 28586 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:32:32.260245 28586 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:44735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:37597,127.2.120.61:44449,127.2.120.60:40595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.62.keytab
--principal=kudu/127.2.120.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.62:37597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.120.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:32:32.261785 28586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:32:32.263273 28586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:32:32.276557 28591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:32.276449 28594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:32.276476 28592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:32:32.277390 28586 server_base.cc:1034] running on GCE node
I20260811 00:32:32.278360 28586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:32:32.279965 28586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:32:32.281241 28586 hybrid_clock.cc:648] HybridClock initialized: now 1786408352281145 us; error 68 us; skew 500 ppm
I20260811 00:32:32.281615 28586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:32:32 dist-test-slave-fd7l krb5kdc[28571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408352, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:32:32.285240 28586 init.cc:377] Logged in from keytab as kudu/127.2.120.62@KRBTEST.COM (short username kudu)
I20260811 00:32:32.287263 28586 webserver.cc:460] Webserver started at http://127.2.120.62:40323/ using document root <none> and password file <none>
I20260811 00:32:32.288139 28586 fs_manager.cc:359] Metadata directory not provided
I20260811 00:32:32.288308 28586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:32:32.288769 28586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:32:32.292335 28586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-0/data/instance:
uuid: "5af25a2921344a719c80fefd75cb5e47"
format_stamp: "Formatted at 2026-08-11 00:32:32 on dist-test-slave-fd7l"
server_key: "4ed0b83b12a4fa4cbeafa305925387ac"
server_key_iv: "9a184d9729a6f247a791ff550c90a4f9"
server_key_version: "encryptionkey@0"
I20260811 00:32:32.293488 28586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-0/wal/instance:
uuid: "5af25a2921344a719c80fefd75cb5e47"
format_stamp: "Formatted at 2026-08-11 00:32:32 on dist-test-slave-fd7l"
server_key: "4ed0b83b12a4fa4cbeafa305925387ac"
server_key_iv: "9a184d9729a6f247a791ff550c90a4f9"
server_key_version: "encryptionkey@0"
I20260811 00:32:32.299533 28586 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260811 00:32:32.303791 28601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:32:32.305688 28586 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260811 00:32:32.305971 28586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-0/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-0/wal
uuid: "5af25a2921344a719c80fefd75cb5e47"
format_stamp: "Formatted at 2026-08-11 00:32:32 on dist-test-slave-fd7l"
server_key: "4ed0b83b12a4fa4cbeafa305925387ac"
server_key_iv: "9a184d9729a6f247a791ff550c90a4f9"
server_key_version: "encryptionkey@0"
I20260811 00:32:32.306262 28586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:32:32.334637 28586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:32:32.339632 28586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:32:32.339990 28586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:32:32.361961 28586 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.62:37597
I20260811 00:32:32.361975 28658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.62:37597 every 8 connection(s)
I20260811 00:32:32.363500 28586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-0/data/info.pb
I20260811 00:32:32.364699 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 28586
I20260811 00:32:32.364908 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-0/wal/instance
I20260811 00:32:32.365481 2528 external_mini_cluster.cc:1428] Setting key 64fa9211388ed0669485892fb879ad86
I20260811 00:32:32.368664 28659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:32:32.379263 28659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:32:32 dist-test-slave-fd7l krb5kdc[28571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408352, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.120.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.61@KRBTEST.COM" created.
I20260811 00:32:32.403739 28665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:32.383104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47778 (local address 127.2.120.62:37597)
0811 00:32:32.385129 (+ 2025us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:32.385138 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:32.385193 (+ 55us) server_negotiation.cc:407] Connection header received
0811 00:32:32.386834 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:32.386840 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:32.387106 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:32.387324 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:32.388720 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:32.389944 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:32.391359 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:32.391945 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:32.392162 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:32.396042 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:32.396051 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:32.396066 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:32.396102 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:32.399172 (+ 3070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:32.399926 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:32.399932 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:32.399934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:32.400095 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:32.400504 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:32.400535 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:32.400538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:32.401138 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:32.401354 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:32.402057 (+ 703us) server_negotiation.cc:299] Negotiation successful
0811 00:32:32.402681 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":274,"server-negotiator.queue_time_us":1791,"thread_start_us":1494,"threads_started":1}
I20260811 00:32:32.403801 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:32.383086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37597 (local address 127.0.0.1:47778)
0811 00:32:32.384105 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:32.384155 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 00:32:32.386413 (+ 2258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:32:32.387337 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:32:32.387344 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:32:32.388104 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:32:32.388445 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:32.388464 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:32.390124 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:32.390129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:32.391210 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:32.391223 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:32.392204 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:32.392212 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:32.392392 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:32.393227 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:32:32.393253 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:32:32.395776 (+ 2523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:32:32.399330 (+ 3554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:32.399342 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:32.399356 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:32.399774 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:32.400212 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:32.400217 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:32.400221 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:32.400387 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:32.401283 (+ 896us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:32:32.401293 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:32:32.401555 (+ 262us) client_negotiation.cc:769] Sending connection context
0811 00:32:32.401829 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 00:32:32.402492 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":1,"thread_start_us":490,"threads_started":1}
I20260811 00:32:32.407076 28659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:32:32.408408 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:32.407896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:44449 (local address 127.0.0.1:46190)
0811 00:32:32.408104 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:32.408246 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:44449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 00:32:32.408739 28605 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.120.61:44449: connect: Connection refused (error 111)
I20260811 00:32:32.410492 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:32.410009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:44449 (local address 127.0.0.1:46196)
0811 00:32:32.410195 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:32.410289 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:44449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 00:32:32.410734 28659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.61:44449: Network error: Client connection negotiation failed: client connection to 127.2.120.61:44449: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.61.keytab.
Entry for principal kudu/127.2.120.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.61.keytab.
I20260811 00:32:32.440594 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.61:44449
--webserver_interface=127.2.120.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.61.keytab
--principal=kudu/127.2.120.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:44735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:37597,127.2.120.61:44449,127.2.120.60:40595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:32:32.460739 28659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } attempt: 1
I20260811 00:32:32.464790 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:32.463385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:44449 (local address 127.0.0.1:46208)
0811 00:32:32.463896 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:32.464239 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:44449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183}
I20260811 00:32:32.468142 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:32.466741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:44449 (local address 127.0.0.1:46210)
0811 00:32:32.467314 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:32.467607 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:44449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260811 00:32:32.468891 28659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.61:44449: Network error: Client connection negotiation failed: client connection to 127.2.120.61:44449: connect: Connection refused (error 111)
I20260811 00:32:32.537357 28659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } attempt: 2
I20260811 00:32:32.538775 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:32.538156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:44449 (local address 127.0.0.1:46216)
0811 00:32:32.538423 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:32.538557 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:44449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 00:32:32.540078 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:32.539566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:44449 (local address 127.0.0.1:46230)
0811 00:32:32.539740 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:32.539860 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:44449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 00:32:32.540371 28659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.61:44449: Network error: Client connection negotiation failed: client connection to 127.2.120.61:44449: connect: Connection refused (error 111)
I20260811 00:32:32.631680 28659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } attempt: 3
I20260811 00:32:32.632995 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:32.632436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:44449 (local address 127.0.0.1:46246)
0811 00:32:32.632701 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:32.632811 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:44449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 00:32:32.634218 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:32.633719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:44449 (local address 127.0.0.1:46258)
0811 00:32:32.633904 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:32.634019 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:44449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 00:32:32.634449 28659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.61:44449: Network error: Client connection negotiation failed: client connection to 127.2.120.61:44449: connect: Connection refused (error 111)
I20260811 00:32:32.777751 28659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } attempt: 4
I20260811 00:32:32.779160 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:32.778478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:44449 (local address 127.0.0.1:46272)
0811 00:32:32.778858 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:32.778973 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:44449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193}
I20260811 00:32:32.780331 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:32.779866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:44449 (local address 127.0.0.1:46274)
0811 00:32:32.780049 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:32.780163 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.61:44449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 00:32:32.780572 28659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.61:44449: Network error: Client connection negotiation failed: client connection to 127.2.120.61:44449: connect: Connection refused (error 111)
W20260811 00:32:32.835856 28668 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 00:32:32.836251 28668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:32:32.836381 28668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:32:32.836488 28668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:32:32.844156 28668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:32:32.844308 28668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:32:32.844409 28668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:32:32.844467 28668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:32:32.844570 28668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:32:32.855078 28668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:44735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:37597,127.2.120.61:44449,127.2.120.60:40595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.61.keytab
--principal=kudu/127.2.120.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.61:44449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.120.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:32:32.856452 28668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:32:32.857795 28668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:32:32.870026 28676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:32.870085 28674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:32.870062 28673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:32:32.871502 28668 server_base.cc:1034] running on GCE node
I20260811 00:32:32.872334 28668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:32:32.873862 28668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:32:32.875155 28668 hybrid_clock.cc:648] HybridClock initialized: now 1786408352875066 us; error 56 us; skew 500 ppm
I20260811 00:32:32.875551 28668 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:32:32 dist-test-slave-fd7l krb5kdc[28571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408352, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:32:32.879210 28668 init.cc:377] Logged in from keytab as kudu/127.2.120.61@KRBTEST.COM (short username kudu)
I20260811 00:32:32.881040 28668 webserver.cc:460] Webserver started at http://127.2.120.61:34475/ using document root <none> and password file <none>
I20260811 00:32:32.881803 28668 fs_manager.cc:359] Metadata directory not provided
I20260811 00:32:32.881942 28668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:32:32.882301 28668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:32:32.885627 28668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-1/data/instance:
uuid: "7689566d31074095b4a9b7f6fa004d59"
format_stamp: "Formatted at 2026-08-11 00:32:32 on dist-test-slave-fd7l"
server_key: "0149092262b088886f489bd66d753eb4"
server_key_iv: "b765c18557975616acbc502f8eb4d215"
server_key_version: "encryptionkey@0"
I20260811 00:32:32.886426 28668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-1/wal/instance:
uuid: "7689566d31074095b4a9b7f6fa004d59"
format_stamp: "Formatted at 2026-08-11 00:32:32 on dist-test-slave-fd7l"
server_key: "0149092262b088886f489bd66d753eb4"
server_key_iv: "b765c18557975616acbc502f8eb4d215"
server_key_version: "encryptionkey@0"
I20260811 00:32:32.891829 28668 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:32:32.895442 28685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:32:32.896914 28668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 00:32:32.897100 28668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-1/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-1/wal
uuid: "7689566d31074095b4a9b7f6fa004d59"
format_stamp: "Formatted at 2026-08-11 00:32:32 on dist-test-slave-fd7l"
server_key: "0149092262b088886f489bd66d753eb4"
server_key_iv: "b765c18557975616acbc502f8eb4d215"
server_key_version: "encryptionkey@0"
I20260811 00:32:32.897325 28668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:32:32.911427 28668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:32:32.916047 28668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:32:32.916329 28668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:32:32.936277 28668 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.61:44449
I20260811 00:32:32.936411 28739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.61:44449 every 8 connection(s)
I20260811 00:32:32.937901 28668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-1/data/info.pb
I20260811 00:32:32.942281 28740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:32:32.946683 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 28668
I20260811 00:32:32.946918 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-1/wal/instance
I20260811 00:32:32.947466 2528 external_mini_cluster.cc:1428] Setting key 2b632308489aa2a24562b1fc475f149e
I20260811 00:32:32.951532 28740 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:32:32 dist-test-slave-fd7l krb5kdc[28571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408352, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.61@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:32:32.980062 28743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:32.955407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47790 (local address 127.2.120.62:37597)
0811 00:32:32.956032 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:32.956040 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:32.957163 (+ 1123us) server_negotiation.cc:407] Connection header received
0811 00:32:32.959369 (+ 2206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:32.959378 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:32.959493 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:32.959628 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:32.962644 (+ 3016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:32.964001 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:32.967364 (+ 3363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:32.968051 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:32.969480 (+ 1429us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:32.973181 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:32.973186 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:32.973189 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:32.973240 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:32.976472 (+ 3232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:32.977447 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:32.977452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:32.977456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:32.977606 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:32.978091 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:32.978097 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:32.978100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:32.978401 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:32.978598 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:32.979413 (+ 815us) server_negotiation.cc:299] Negotiation successful
0811 00:32:32.979658 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":298,"threads_started":1}
I20260811 00:32:32.981172 28744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:32.956033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37597 (local address 127.0.0.1:47790)
0811 00:32:32.956957 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:32.957015 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 00:32:32.958996 (+ 1981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:32:32.960196 (+ 1200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:32:32.960212 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:32:32.961261 (+ 1049us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:32:32.962289 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:32.962327 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:32.964265 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:32.964274 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:32.967006 (+ 2732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:32.967059 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:32.968386 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:32.968400 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:32.968894 (+ 494us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:32.969952 (+ 1058us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:32:32.969998 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:32:32.972838 (+ 2840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:32:32.976723 (+ 3885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:32.976735 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:32.976763 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:32.977290 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:32.977739 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:32.977744 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:32.977747 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:32.977954 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:32.978557 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:32:32.978567 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:32:32.979004 (+ 437us) client_negotiation.cc:769] Sending connection context
0811 00:32:32.979299 (+ 295us) client_negotiation.cc:240] Negotiation successful
0811 00:32:32.979890 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":420,"threads_started":1}
I20260811 00:32:32.983507 28740 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.120.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.120.60@KRBTEST.COM" created.
Aug 11 00:32:32 dist-test-slave-fd7l krb5kdc[28571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408352, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.61@KRBTEST.COM for kudu/127.2.120.61@KRBTEST.COM
I20260811 00:32:33.002269 28744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:32.984498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:44449 (local address 127.0.0.1:46284)
0811 00:32:32.984776 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:32.984794 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:32:32.984906 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:32:32.986468 (+ 1562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:32:32.986473 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:32:32.987039 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:32:32.987192 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:32.987202 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:32.989280 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:32.989288 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:32.990252 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:32.990266 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:32.991295 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:32.991301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:32.991495 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:32.992212 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:32:32.992232 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:32:32.994839 (+ 2607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:32:32.998752 (+ 3913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:32.998761 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:32.998767 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:32.999283 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:32.999732 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:32.999739 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:32.999743 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:32.999911 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:33.001123 (+ 1212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:32:33.001130 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:32:33.001285 (+ 155us) client_negotiation.cc:769] Sending connection context
0811 00:32:33.001440 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 00:32:33.001768 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260811 00:32:33.002318 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:32.984747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46284 (local address 127.2.120.61:44449)
0811 00:32:32.985918 (+ 1171us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:32.985926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:32.985959 (+ 33us) server_negotiation.cc:407] Connection header received
0811 00:32:32.986021 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:32.986029 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:32.986259 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:32.986483 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:32.987406 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:32.989116 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:32.990451 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:32.991107 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:32.991263 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:32.995104 (+ 3841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:32.995121 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:32.995141 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:32.995176 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:32.998552 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:32.999453 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:32.999461 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:32.999464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:32.999606 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.000218 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.000229 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.000234 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.000966 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:33.001212 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:33.001661 (+ 449us) server_negotiation.cc:299] Negotiation successful
0811 00:32:33.001901 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":977,"thread_start_us":720,"threads_started":1}
I20260811 00:32:33.004935 28740 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:32:33.006366 28744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.005797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:32862)
0811 00:32:33.006033 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.006188 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260811 00:32:33.006717 28688 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
I20260811 00:32:33.008801 28744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.008246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:32878)
0811 00:32:33.008462 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.008629 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 00:32:33.009083 28740 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:40595: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.120.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.60.keytab.
Entry for principal kudu/127.2.120.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.60.keytab.
I20260811 00:32:33.031790 2528 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY6MNab/build/asan/bin/kudu
/tmp/dist-test-taskY6MNab/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.60:40595
--webserver_interface=127.2.120.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.60.keytab
--principal=kudu/127.2.120.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.120.20:44735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.120.62:37597,127.2.120.61:44449,127.2.120.60:40595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:32:33.058410 28740 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } attempt: 1
I20260811 00:32:33.059945 28744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.059234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:32880)
0811 00:32:33.059570 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.059721 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 00:32:33.061247 28744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.060739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:32892)
0811 00:32:33.060951 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.061066 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 00:32:33.061487 28740 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:40595: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
I20260811 00:32:33.079845 28659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } attempt: 5
Aug 11 00:32:33 dist-test-slave-fd7l krb5kdc[28571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408352, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for kudu/127.2.120.61@KRBTEST.COM
I20260811 00:32:33.100368 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.080832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46298 (local address 127.2.120.61:44449)
0811 00:32:33.081028 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:33.081033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:33.081056 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:32:33.081317 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:33.081321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:33.081405 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:33.081511 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:33.082601 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.084113 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.085936 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.086912 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.087089 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:33.092112 (+ 5023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:33.092122 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:33.092131 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:33.092200 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:33.096757 (+ 4557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.097823 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.097835 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.097842 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.098021 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.098531 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.098537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.098540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.098981 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:33.099153 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:33.099779 (+ 626us) server_negotiation.cc:299] Negotiation successful
0811 00:32:33.100050 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 00:32:33.100373 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.080740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:44449 (local address 127.0.0.1:46298)
0811 00:32:33.080973 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.080998 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:32:33.081141 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:32:33.081597 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:32:33.081602 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:32:33.082185 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:32:33.082416 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:33.082430 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.084320 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.084327 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:33.085699 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:33.085718 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.087108 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.087115 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:33.087363 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:33.088199 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:32:33.088223 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:32:33.091699 (+ 3476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:32:33.096997 (+ 5298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:33.097004 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:33.097010 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:33.097658 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:33.098164 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:33.098170 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:33.098173 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:33.098365 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:33.099140 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:32:33.099145 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:32:33.099400 (+ 255us) client_negotiation.cc:769] Sending connection context
0811 00:32:33.099626 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 00:32:33.099970 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 00:32:33.102213 28659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:32:33.103659 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.103088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:32900)
0811 00:32:33.103320 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.103433 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 00:32:33.104784 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.104343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:32916)
0811 00:32:33.104502 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.104630 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 00:32:33.105032 28659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:40595: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
I20260811 00:32:33.129747 28740 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } attempt: 2
I20260811 00:32:33.131109 28744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.130543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:32932)
0811 00:32:33.130792 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.130903 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 00:32:33.132387 28744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.131856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:32934)
0811 00:32:33.132064 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.132176 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 00:32:33.132707 28740 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:40595: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
I20260811 00:32:33.156314 28659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } attempt: 1
I20260811 00:32:33.157655 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.157150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:32950)
0811 00:32:33.157367 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.157467 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 00:32:33.158762 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.158394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:32962)
0811 00:32:33.158572 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.158654 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 00:32:33.158989 28659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:40595: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
I20260811 00:32:33.223989 28740 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } attempt: 3
I20260811 00:32:33.225414 28744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.224805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:32972)
0811 00:32:33.225113 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.225229 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 00:32:33.226855 28744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.226317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:32984)
0811 00:32:33.226593 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.226715 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
W20260811 00:32:33.227176 28740 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:40595: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
I20260811 00:32:33.227401 28659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } attempt: 2
I20260811 00:32:33.229092 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.228428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:32990)
0811 00:32:33.228764 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.228887 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
I20260811 00:32:33.230324 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.229915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:33004)
0811 00:32:33.230099 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.230196 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 00:32:33.230656 28659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:40595: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
I20260811 00:32:33.336938 28659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } attempt: 3
I20260811 00:32:33.338371 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.337762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:44174)
0811 00:32:33.338013 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.338157 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 00:32:33.339808 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.339200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:44188)
0811 00:32:33.339494 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.339628 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260811 00:32:33.340178 28659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:40595: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
I20260811 00:32:33.370451 28740 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } attempt: 4
I20260811 00:32:33.371809 28744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.371234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:44194)
0811 00:32:33.371539 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.371633 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 00:32:33.372954 28744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.372591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:44196)
0811 00:32:33.372752 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.372839 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 00:32:33.373193 28740 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:40595: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
W20260811 00:32:33.444372 28749 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 00:32:33.444845 28749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:32:33.444983 28749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:32:33.445130 28749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:32:33.453321 28749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:32:33.453444 28749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:32:33.453555 28749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:32:33.453642 28749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:32:33.453740 28749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:32:33.466109 28749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.120.20:44735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.120.62:37597,127.2.120.61:44449,127.2.120.60:40595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/krb5kdc/kudu_127.2.120.60.keytab
--principal=kudu/127.2.120.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.120.60:40595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.120.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:32:33.468065 28749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:32:33.469969 28749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:32:33.483912 28754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:33.483899 28757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:32:33.483986 28755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:32:33.485250 28749 server_base.cc:1034] running on GCE node
I20260811 00:32:33.486143 28749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:32:33.488075 28749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:32:33.489382 28749 hybrid_clock.cc:648] HybridClock initialized: now 1786408353489250 us; error 109 us; skew 500 ppm
I20260811 00:32:33.489823 28749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:32:33 dist-test-slave-fd7l krb5kdc[28571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408353, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:32:33.499089 28749 init.cc:377] Logged in from keytab as kudu/127.2.120.60@KRBTEST.COM (short username kudu)
I20260811 00:32:33.501106 28749 webserver.cc:460] Webserver started at http://127.2.120.60:45519/ using document root <none> and password file <none>
I20260811 00:32:33.501919 28749 fs_manager.cc:359] Metadata directory not provided
I20260811 00:32:33.502025 28749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:32:33.502363 28749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:32:33.505782 28749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-2/data/instance:
uuid: "e93a5682d58a468c88162d0029fd67c1"
format_stamp: "Formatted at 2026-08-11 00:32:33 on dist-test-slave-fd7l"
server_key: "642fe3ab15397f1a61f17d7772840880"
server_key_iv: "0e2d2e848d27282a8088da91bfdacca1"
server_key_version: "encryptionkey@0"
I20260811 00:32:33.506601 28749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-2/wal/instance:
uuid: "e93a5682d58a468c88162d0029fd67c1"
format_stamp: "Formatted at 2026-08-11 00:32:33 on dist-test-slave-fd7l"
server_key: "642fe3ab15397f1a61f17d7772840880"
server_key_iv: "0e2d2e848d27282a8088da91bfdacca1"
server_key_version: "encryptionkey@0"
I20260811 00:32:33.512740 28749 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260811 00:32:33.516465 28765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:32:33.517478 28659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } attempt: 4
I20260811 00:32:33.518285 28749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:32:33.518507 28749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-2/data,/tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-2/wal
uuid: "e93a5682d58a468c88162d0029fd67c1"
format_stamp: "Formatted at 2026-08-11 00:32:33 on dist-test-slave-fd7l"
server_key: "642fe3ab15397f1a61f17d7772840880"
server_key_iv: "0e2d2e848d27282a8088da91bfdacca1"
server_key_version: "encryptionkey@0"
I20260811 00:32:33.518726 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.518174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:44206)
0811 00:32:33.518416 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.518532 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 00:32:33.518777 28749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:32:33.520387 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.519985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:44214)
0811 00:32:33.520149 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.520235 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 00:32:33.520694 28659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.120.60:40595: Network error: Client connection negotiation failed: client connection to 127.2.120.60:40595: connect: Connection refused (error 111)
I20260811 00:32:33.537177 28749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:32:33.543093 28749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:32:33.543447 28749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:32:33.563612 28749 rpc_server.cc:307] RPC server started. Bound to: 127.2.120.60:40595
I20260811 00:32:33.563666 28822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.120.60:40595 every 8 connection(s)
I20260811 00:32:33.565073 28749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-2/data/info.pb
I20260811 00:32:33.569090 28823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:32:33.569363 2528 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY6MNab/build/asan/bin/kudu as pid 28749
I20260811 00:32:33.569555 2528 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY6MNab/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407939260739-2528-0/minicluster-data/master-2/wal/instance
I20260811 00:32:33.570042 2528 external_mini_cluster.cc:1428] Setting key 4e05c9813f1355304bdb575d58ae22aa
I20260811 00:32:33.580106 28823 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:32:33 dist-test-slave-fd7l krb5kdc[28571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.60@KRBTEST.COM
Aug 11 00:32:33 dist-test-slave-fd7l krb5kdc[28571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408353, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.60@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
I20260811 00:32:33.611758 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.571970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44230 (local address 127.2.120.60:40595)
0811 00:32:33.572927 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:33.572943 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:33.572986 (+ 43us) server_negotiation.cc:407] Connection header received
0811 00:32:33.574871 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:33.574902 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:33.575407 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:33.575840 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:33.577904 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.579233 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.581152 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.582256 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.582467 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:33.588360 (+ 5893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:33.588369 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:33.588384 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:33.588442 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:33.595278 (+ 6836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.598096 (+ 2818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.598129 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.598160 (+ 31us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.598771 (+ 611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.600055 (+ 1284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.600065 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.600071 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.602863 (+ 2792us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:33.603847 (+ 984us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:33.605385 (+ 1538us) server_negotiation.cc:299] Negotiation successful
0811 00:32:33.606638 (+ 1253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":483,"threads_started":1}
I20260811 00:32:33.618553 28827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.583701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56498 (local address 127.2.120.62:37597)
0811 00:32:33.584348 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:33.584357 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:33.585068 (+ 711us) server_negotiation.cc:407] Connection header received
0811 00:32:33.585300 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:33.585307 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:33.585400 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:33.585529 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:33.587406 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.588583 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.591903 (+ 3320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.593188 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.593502 (+ 314us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:33.605027 (+ 11525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:33.605041 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:33.605046 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:33.605147 (+ 101us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:33.613811 (+ 8664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.615638 (+ 1827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.615647 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.615650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.615952 (+ 302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.616722 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.616728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.616732 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.617141 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:33.617331 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:33.617987 (+ 656us) server_negotiation.cc:299] Negotiation successful
0811 00:32:33.618230 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":297,"threads_started":1}
I20260811 00:32:33.618942 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.583729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.62:37597 (local address 127.0.0.1:56498)
0811 00:32:33.584873 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.584944 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 00:32:33.585142 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:32:33.585610 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:32:33.585615 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:32:33.586759 (+ 1144us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:32:33.587191 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:33.587211 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.588785 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.588792 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:33.590991 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:33.591039 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.593823 (+ 2784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.593843 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:33.594305 (+ 462us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:33.597030 (+ 2725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:32:33.597105 (+ 75us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:32:33.604294 (+ 7189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:32:33.614340 (+ 10046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:33.614380 (+ 40us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:33.614397 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:33.615269 (+ 872us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:33.616296 (+ 1027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:33.616302 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:33.616306 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:33.616493 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:33.617312 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:32:33.617320 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:32:33.617553 (+ 233us) client_negotiation.cc:769] Sending connection context
0811 00:32:33.617791 (+ 238us) client_negotiation.cc:240] Negotiation successful
0811 00:32:33.618328 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"thread_start_us":632,"threads_started":1}
I20260811 00:32:33.620891 28823 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:32:33 dist-test-slave-fd7l krb5kdc[28571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.62@KRBTEST.COM
Aug 11 00:32:33 dist-test-slave-fd7l krb5kdc[28571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408353, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.60@KRBTEST.COM for kudu/127.2.120.61@KRBTEST.COM
I20260811 00:32:33.637756 28827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.617630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56502 (local address 127.2.120.62:37597)
0811 00:32:33.618903 (+ 1273us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:33.618909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:33.618925 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:32:33.618977 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:33.618983 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:33.619132 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:33.619254 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:33.621077 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.621879 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.623683 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.624277 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.624423 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:33.629550 (+ 5127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:33.629560 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:33.629566 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:33.629616 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:33.633484 (+ 3868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.634386 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.634392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.634395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.634557 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.635140 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.635238 (+ 98us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.635244 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.636108 (+ 864us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:33.636692 (+ 584us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:33.637071 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 00:32:33.637359 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1178,"thread_start_us":1673,"threads_started":1}
I20260811 00:32:33.638728 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.622179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.61:44449 (local address 127.0.0.1:44146)
0811 00:32:33.622421 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.622445 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:32:33.622638 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:32:33.623297 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:32:33.623304 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:32:33.623858 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:32:33.624077 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:33.624089 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.625374 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.625381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:33.626295 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:33.626307 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.627055 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.627062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:33.627242 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:33.627826 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:32:33.627849 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:32:33.630533 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:32:33.634470 (+ 3937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:33.634475 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:33.634477 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:33.635088 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:33.636050 (+ 962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:33.636056 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:33.636060 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:33.636511 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:33.637389 (+ 878us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:32:33.637424 (+ 35us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:32:33.637759 (+ 335us) client_negotiation.cc:769] Sending connection context
0811 00:32:33.638134 (+ 375us) client_negotiation.cc:240] Negotiation successful
0811 00:32:33.638474 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 00:32:33.638949 28833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.622169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44146 (local address 127.2.120.61:44449)
0811 00:32:33.622987 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:33.622992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:33.623008 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:32:33.623059 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:33.623063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:33.623126 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:33.623257 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:33.624309 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.625226 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.626451 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.626876 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.627021 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:33.630794 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:33.630803 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:33.630810 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:33.630861 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:33.634255 (+ 3394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.635345 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.635356 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.635361 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.635606 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.636732 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.636740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.636745 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.637195 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:33.637385 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:33.638223 (+ 838us) server_negotiation.cc:299] Negotiation successful
0811 00:32:33.638491 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":530,"threads_started":1}
I20260811 00:32:33.639973 28823 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:32:33 dist-test-slave-fd7l krb5kdc[28571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408353, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.60@KRBTEST.COM for kudu/127.2.120.60@KRBTEST.COM
Aug 11 00:32:33 dist-test-slave-fd7l krb5kdc[28571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.120.61@KRBTEST.COM
I20260811 00:32:33.658356 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.640818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:44246)
0811 00:32:33.641172 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.641194 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:32:33.641302 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:32:33.641807 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:32:33.641813 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:32:33.642356 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:32:33.642580 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:33.642594 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.644060 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.644067 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:33.645661 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:33.645678 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.646835 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.646841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:33.647082 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:33.648044 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:32:33.648072 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:32:33.651342 (+ 3270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:32:33.655273 (+ 3931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:33.655277 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:33.655280 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:33.655722 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:33.656168 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:33.656173 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:33.656178 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:33.656351 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:33.657194 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:32:33.657200 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:32:33.657398 (+ 198us) client_negotiation.cc:769] Sending connection context
0811 00:32:33.657635 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 00:32:33.657933 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260811 00:32:33.658641 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.641141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44246 (local address 127.2.120.60:40595)
0811 00:32:33.641497 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:33.641501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:33.641513 (+ 12us) server_negotiation.cc:407] Connection header received
0811 00:32:33.641545 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:33.641548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:33.641643 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:33.641714 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:33.642752 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.643817 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.645862 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.646609 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.646738 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:33.651679 (+ 4941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:33.651685 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:33.651688 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:33.651722 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:33.655081 (+ 3359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.655858 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.655863 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.655866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.656028 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.656576 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.656581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.656593 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.657025 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:33.657184 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:33.657752 (+ 568us) server_negotiation.cc:299] Negotiation successful
0811 00:32:33.658210 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":267}
I20260811 00:32:33.659881 28833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.640151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44156 (local address 127.2.120.61:44449)
0811 00:32:33.640322 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:33.640351 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:33.640392 (+ 41us) server_negotiation.cc:407] Connection header received
0811 00:32:33.640449 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:33.640455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:33.640579 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:33.640683 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:33.642685 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.643866 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.645878 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.646578 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.646738 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:33.652662 (+ 5924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:33.652668 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:33.652671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:33.652712 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:33.655889 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.656849 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.656855 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.656861 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.657024 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.657578 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.657581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.657583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.657973 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:33.658243 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:33.659274 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0811 00:32:33.659535 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:32:33.664253 28823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1: Bootstrap starting.
I20260811 00:32:33.665385 2528 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:32:33.668298 28823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1: Neither blocks nor log segments found. Creating new log.
I20260811 00:32:33.669636 28823 log.cc:824] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1: Log is configured to *not* fsync() on all Append() calls
I20260811 00:32:33.668716 28632 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5af25a2921344a719c80fefd75cb5e47"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56502
I20260811 00:32:33.672473 28740 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } attempt: 5
I20260811 00:32:33.673087 28823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1: No bootstrap required, opened a new log
Aug 11 00:32:33 dist-test-slave-fd7l krb5kdc[28571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408352, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.61@KRBTEST.COM for kudu/127.2.120.60@KRBTEST.COM
I20260811 00:32:33.684763 28823 raft_consensus.cc:348] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } } peers { permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } } peers { permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } }
I20260811 00:32:33.685230 28823 raft_consensus.cc:729] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e93a5682d58a468c88162d0029fd67c1, State: Initialized, Role: FOLLOWER
I20260811 00:32:33.685829 28823 consensus_queue.cc:260] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } } peers { permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } } peers { permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } }
I20260811 00:32:33.687101 28835 sys_catalog.cc:455] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } } peers { permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } } peers { permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } } }
I20260811 00:32:33.687541 28835 sys_catalog.cc:458] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:32:33.688194 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.673386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44254 (local address 127.2.120.60:40595)
0811 00:32:33.673667 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:33.673673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:33.673694 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:32:33.673790 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:33.673798 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:33.673902 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:33.674015 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:33.675003 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.675832 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.677105 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.677641 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.677786 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:33.681478 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:33.681485 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:33.681493 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:33.681537 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:33.684595 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.685472 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.685478 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.685481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.685614 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.686132 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.686138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.686141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.686668 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:33.686856 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:33.687493 (+ 637us) server_negotiation.cc:299] Negotiation successful
0811 00:32:33.687841 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:32:33.688375 28823 sys_catalog.cc:564] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:32:33.688688 28744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.673243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:44254)
0811 00:32:33.673498 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.673522 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:32:33.673660 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:32:33.674062 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:32:33.674069 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:32:33.674644 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:32:33.674851 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:33.674861 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.675991 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.675999 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:33.676948 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:33.676960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.677803 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.677809 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:33.678059 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:33.678700 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:32:33.678718 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:32:33.681232 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:32:33.684729 (+ 3497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:33.684737 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:33.684743 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:33.685309 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:33.685767 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:33.685772 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:33.685776 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:33.685961 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:33.686927 (+ 966us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:32:33.686932 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:32:33.687084 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 00:32:33.687252 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 00:32:33.688033 (+ 781us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 00:32:33.694787 28740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59: Bootstrap starting.
I20260811 00:32:33.699010 28740 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59: Neither blocks nor log segments found. Creating new log.
I20260811 00:32:33.700696 28740 log.cc:824] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59: Log is configured to *not* fsync() on all Append() calls
I20260811 00:32:33.706604 28740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59: No bootstrap required, opened a new log
W20260811 00:32:33.707603 28847 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:32:33.707762 28847 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:32:33.699563 28749 master_runner.cc:186] Error getting master registration for 127.2.120.62:37597: OK, instance_id { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" instance_seqno: 1786408352355343 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 00:32:33.711189 28749 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 00:32:33.718508 28740 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } } peers { permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } } peers { permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } }
I20260811 00:32:33.718938 28740 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7689566d31074095b4a9b7f6fa004d59, State: Initialized, Role: FOLLOWER
I20260811 00:32:33.719508 28740 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } } peers { permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } } peers { permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } }
I20260811 00:32:33.720957 28850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } } peers { permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } } peers { permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } } }
I20260811 00:32:33.721724 28850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:32:33.722191 28740 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 00:32:33.744262 28863 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:32:33.744676 28863 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:32:33.732722 28668 master_runner.cc:186] Error getting master registration for 127.2.120.62:37597: OK, instance_id { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" instance_seqno: 1786408352355343 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 00:32:33.746291 28668 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 00:32:33.795885 28773 tablet.cc:2369] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:32:33.797977 28659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } attempt: 5
Aug 11 00:32:33 dist-test-slave-fd7l krb5kdc[28571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786408352, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.120.62@KRBTEST.COM for kudu/127.2.120.60@KRBTEST.COM
I20260811 00:32:33.818385 28663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.798797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.120.60:40595 (local address 127.0.0.1:44270)
0811 00:32:33.799118 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 00:32:33.799139 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:32:33.799266 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:32:33.799741 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:32:33.799746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:32:33.800295 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:32:33.800547 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:33.800560 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.802499 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.802507 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:33.804042 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:32:33.804062 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.805339 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.805347 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:32:33.805575 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:33.806461 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:32:33.806490 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:32:33.809885 (+ 3395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:32:33.814795 (+ 4910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:33.814804 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:33.814810 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:33.815417 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:33.816005 (+ 588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:32:33.816010 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:32:33.816014 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:32:33.816253 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:32:33.817139 (+ 886us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:32:33.817145 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:32:33.817354 (+ 209us) client_negotiation.cc:769] Sending connection context
0811 00:32:33.817534 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 00:32:33.817874 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260811 00:32:33.820016 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.799054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44270 (local address 127.2.120.60:40595)
0811 00:32:33.799351 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:33.799358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:33.799385 (+ 27us) server_negotiation.cc:407] Connection header received
0811 00:32:33.799454 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:33.799459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:33.799547 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:33.799687 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:33.800788 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.802330 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.804253 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.805161 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.805327 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:33.810157 (+ 4830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:33.810166 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:33.810171 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:33.810226 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:33.814583 (+ 4357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.815651 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.815658 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.815662 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.815849 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.816410 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.816417 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.816429 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.816876 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:33.817062 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:33.819131 (+ 2069us) server_negotiation.cc:299] Negotiation successful
0811 00:32:33.819422 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260811 00:32:33.827549 28659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47: Bootstrap starting.
I20260811 00:32:33.833010 28659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47: Neither blocks nor log segments found. Creating new log.
I20260811 00:32:33.834848 28659 log.cc:824] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47: Log is configured to *not* fsync() on all Append() calls
I20260811 00:32:33.841084 28659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47: No bootstrap required, opened a new log
I20260811 00:32:33.856284 28659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } } peers { permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } } peers { permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } }
I20260811 00:32:33.857354 28659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af25a2921344a719c80fefd75cb5e47, State: Initialized, Role: FOLLOWER
I20260811 00:32:33.858453 28659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } } peers { permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } } peers { permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } }
I20260811 00:32:33.860312 28866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } } peers { permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } } peers { permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } } }
I20260811 00:32:33.861078 28866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:32:33.864476 28632 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 00:32:33.864784 28632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:32:33.865027 28659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:32:33.868891 28632 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } } peers { permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } } peers { permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } }
I20260811 00:32:33.871860 28713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5af25a2921344a719c80fefd75cb5e47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7689566d31074095b4a9b7f6fa004d59"
I20260811 00:32:33.872488 28713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:32:33.877749 28713 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5af25a2921344a719c80fefd75cb5e47 in term 1.
I20260811 00:32:33.878757 28605 leader_election.cc:302] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af25a2921344a719c80fefd75cb5e47, 7689566d31074095b4a9b7f6fa004d59; no voters:
I20260811 00:32:33.879475 28866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:32:33.883136 28632 leader_election.cc:288] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [CANDIDATE]: Term 1 election: Requested vote from peers 7689566d31074095b4a9b7f6fa004d59 (127.2.120.61:44449), e93a5682d58a468c88162d0029fd67c1 (127.2.120.60:40595)
I20260811 00:32:33.884414 28796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5af25a2921344a719c80fefd75cb5e47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e93a5682d58a468c88162d0029fd67c1"
I20260811 00:32:33.885076 28796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:32:33.892118 28866 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [term 1 LEADER]: Becoming Leader. State: Replica: 5af25a2921344a719c80fefd75cb5e47, State: Running, Role: LEADER
I20260811 00:32:33.892660 28796 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5af25a2921344a719c80fefd75cb5e47 in term 1.
I20260811 00:32:33.892963 28866 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } } peers { permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } } peers { permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } }
I20260811 00:32:33.909850 28868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5af25a2921344a719c80fefd75cb5e47. Latest consensus state: current_term: 1 leader_uuid: "5af25a2921344a719c80fefd75cb5e47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } } peers { permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } } peers { permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } } }
I20260811 00:32:33.910403 28868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:33.919466 28879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260811 00:32:33.919471 28691 tablet.cc:2369] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:32:33.940485 28879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:32:33.961923 28796 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1 [term 1 FOLLOWER]: Refusing update from remote peer 5af25a2921344a719c80fefd75cb5e47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:32:33.963294 28868 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:32:33.970887 28713 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59 [term 1 FOLLOWER]: Refusing update from remote peer 5af25a2921344a719c80fefd75cb5e47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:32:33.971995 28868 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:32:33.984500 28895 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 00:32:33.985817 28833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.915995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44166 (local address 127.2.120.61:44449)
0811 00:32:33.916178 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:33.916184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:33.926730 (+ 10546us) server_negotiation.cc:407] Connection header received
0811 00:32:33.929108 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:33.929116 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:33.929215 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:33.929294 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:33.940933 (+ 11639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.942160 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.951492 (+ 9332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.952326 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.952462 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:33.965444 (+ 12982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:33.965463 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:33.965469 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:33.965565 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:33.969740 (+ 4175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.977795 (+ 8055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.977802 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.977808 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.977986 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.981571 (+ 3585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.981579 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.981583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.982023 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:33.982356 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:33.985080 (+ 2724us) server_negotiation.cc:299] Negotiation successful
0811 00:32:33.985382 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:32:33.987236 28827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.914916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56508 (local address 127.2.120.62:37597)
0811 00:32:33.915123 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:33.915131 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:33.928646 (+ 13515us) server_negotiation.cc:407] Connection header received
0811 00:32:33.931734 (+ 3088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:33.931742 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:33.931841 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:33.931917 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:33.938066 (+ 6149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.939320 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.953897 (+ 14577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.954831 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.954945 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:33.971018 (+ 16073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:33.971026 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:33.971032 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:33.971090 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:33.975106 (+ 4016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.978648 (+ 3542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.978655 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.978659 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.978817 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.982918 (+ 4101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.982924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.982928 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.983333 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:33.983572 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:33.986527 (+ 2955us) server_negotiation.cc:299] Negotiation successful
0811 00:32:33.986792 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 00:32:33.996866 28835 sys_catalog.cc:455] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5af25a2921344a719c80fefd75cb5e47. Latest consensus state: current_term: 1 leader_uuid: "5af25a2921344a719c80fefd75cb5e47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } } peers { permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } } peers { permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } } }
I20260811 00:32:33.997303 28835 sys_catalog.cc:458] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:32:33.999250 28894 mvcc.cc:205] Tried to move back new op lower bound from 7317128617812717568 to 7317128617552424960. Current Snapshot: MvccSnapshot[applied={T|T < 7317128617812717568}]
I20260811 00:32:34.001184 28889 mvcc.cc:205] Tried to move back new op lower bound from 7317128617812717568 to 7317128617552424960. Current Snapshot: MvccSnapshot[applied={T|T < 7317128617812717568}]
I20260811 00:32:34.002547 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:33.927304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44274 (local address 127.2.120.60:40595)
0811 00:32:33.927528 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:33.927539 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:33.947003 (+ 19464us) server_negotiation.cc:407] Connection header received
0811 00:32:33.948665 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:33.948673 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:33.948807 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:33.948895 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:33.961376 (+ 12481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.962620 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.979459 (+ 16839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:33.980251 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:33.980373 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:33.992807 (+ 12434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:32:33.992815 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:32:33.992820 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:32:33.992866 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:32:33.998203 (+ 5337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:33.999430 (+ 1227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:33.999436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:33.999440 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:33.999688 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:32:34.000243 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:32:34.000248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:32:34.000251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:32:34.000745 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:32:34.000998 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:34.001835 (+ 837us) server_negotiation.cc:299] Negotiation successful
0811 00:32:34.002104 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:32:34.011911 28866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5af25a2921344a719c80fefd75cb5e47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } } peers { permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } } peers { permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } } }
I20260811 00:32:34.012392 28866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:34.018949 28835 sys_catalog.cc:455] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5af25a2921344a719c80fefd75cb5e47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } } peers { permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } } peers { permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } } }
I20260811 00:32:34.019366 28835 sys_catalog.cc:458] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:32:34.026295 28879 catalog_manager.cc:1347] Generated new cluster ID: 940fed5ae0db4c59be1bf16ae1d6ef98
I20260811 00:32:34.026430 28879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:32:34.047253 28850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5af25a2921344a719c80fefd75cb5e47. Latest consensus state: current_term: 1 leader_uuid: "5af25a2921344a719c80fefd75cb5e47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } } peers { permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } } peers { permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } } }
I20260811 00:32:34.047590 28850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:32:34.051733 28868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5af25a2921344a719c80fefd75cb5e47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } } peers { permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } } peers { permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } } }
I20260811 00:32:34.052035 28868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47 [sys.catalog]: This master's current role is: LEADER
I20260811 00:32:34.061086 28850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5af25a2921344a719c80fefd75cb5e47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af25a2921344a719c80fefd75cb5e47" member_type: VOTER last_known_addr { host: "127.2.120.62" port: 37597 } } peers { permanent_uuid: "7689566d31074095b4a9b7f6fa004d59" member_type: VOTER last_known_addr { host: "127.2.120.61" port: 44449 } } peers { permanent_uuid: "e93a5682d58a468c88162d0029fd67c1" member_type: VOTER last_known_addr { host: "127.2.120.60" port: 40595 } } }
I20260811 00:32:34.061389 28850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:32:34.095790 28879 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:32:34.098095 28879 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:32:34.121379 28879 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5af25a2921344a719c80fefd75cb5e47: Generated new TSK 0
I20260811 00:32:34.122704 28879 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 00:32:34.266719 28827 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:34.253776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56516 (local address 127.2.120.62:37597)
0811 00:32:34.254055 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:34.254061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:34.263115 (+ 9054us) server_negotiation.cc:407] Connection header received
0811 00:32:34.264546 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:34.264554 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:34.264686 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:34.264773 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:34.266411 (+ 1638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56516: BlockingRecv error: recv got EOF from 127.0.0.1:56516 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260811 00:32:34.272342 28826 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:34.268864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44288 (local address 127.2.120.60:40595)
0811 00:32:34.269053 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:34.269060 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:34.269864 (+ 804us) server_negotiation.cc:407] Connection header received
0811 00:32:34.270290 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:34.270296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:34.270385 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:34.270459 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:34.272075 (+ 1616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44288: BlockingRecv error: recv got EOF from 127.0.0.1:44288 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 00:32:34.274540 28833 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:34.257830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44168 (local address 127.2.120.61:44449)
0811 00:32:34.258047 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:34.258054 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:34.268186 (+ 10132us) server_negotiation.cc:407] Connection header received
0811 00:32:34.270815 (+ 2629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:34.270821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:34.270911 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:34.270983 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:34.274151 (+ 3168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44168: BlockingRecv error: recv got EOF from 127.0.0.1:44168 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 00:32:34.326153 28833 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:34.315261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44174 (local address 127.2.120.61:44449)
0811 00:32:34.315712 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:34.315722 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:34.323350 (+ 7628us) server_negotiation.cc:407] Connection header received
0811 00:32:34.324120 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:34.324127 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:34.324270 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:34.324350 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:34.325815 (+ 1465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44174: BlockingRecv error: recv got EOF from 127.0.0.1:44174 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
W20260811 00:32:34.334117 28833 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:34.331945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44186 (local address 127.2.120.61:44449)
0811 00:32:34.332182 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:34.332190 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:34.332332 (+ 142us) server_negotiation.cc:407] Connection header received
0811 00:32:34.332632 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:34.332636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:34.332753 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:34.332869 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:34.333873 (+ 1004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44186: BlockingRecv error: recv got EOF from 127.0.0.1:44186 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260811 00:32:34.342491 28833 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:34.339717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44194 (local address 127.2.120.61:44449)
0811 00:32:34.340009 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:34.340023 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:34.340793 (+ 770us) server_negotiation.cc:407] Connection header received
0811 00:32:34.341070 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:34.341075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:34.341172 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:34.341304 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:34.342189 (+ 885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44194: BlockingRecv error: recv got EOF from 127.0.0.1:44194 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260811 00:32:34.357589 28833 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:34.354849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44196 (local address 127.2.120.61:44449)
0811 00:32:34.355268 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:34.355279 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:34.355888 (+ 609us) server_negotiation.cc:407] Connection header received
0811 00:32:34.356153 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:34.356158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:34.356242 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:34.356389 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:34.357289 (+ 900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44196: BlockingRecv error: recv got EOF from 127.0.0.1:44196 (error 108)
Metrics: {"server-negotiator.queue_time_us":162}
W20260811 00:32:34.402554 28833 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:34.399539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44212 (local address 127.2.120.61:44449)
0811 00:32:34.399775 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:34.399784 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:34.400557 (+ 773us) server_negotiation.cc:407] Connection header received
0811 00:32:34.400917 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:34.400923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:34.401181 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:34.401307 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:34.402260 (+ 953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44212: BlockingRecv error: recv got EOF from 127.0.0.1:44212 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260811 00:32:34.424994 28833 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:34.422309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44228 (local address 127.2.120.61:44449)
0811 00:32:34.422507 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:34.422514 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:34.423346 (+ 832us) server_negotiation.cc:407] Connection header received
0811 00:32:34.423584 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:34.423590 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:34.423682 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:34.423822 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:34.424723 (+ 901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44228: BlockingRecv error: recv got EOF from 127.0.0.1:44228 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 00:32:34.463230 28833 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:34.460773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44230 (local address 127.2.120.61:44449)
0811 00:32:34.461010 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:34.461018 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:34.461435 (+ 417us) server_negotiation.cc:407] Connection header received
0811 00:32:34.461614 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:34.461622 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:34.461771 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:34.461909 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:34.462947 (+ 1038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44230: BlockingRecv error: recv got EOF from 127.0.0.1:44230 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260811 00:32:34.533468 28833 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:34.531056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44234 (local address 127.2.120.61:44449)
0811 00:32:34.531277 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:34.531287 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:34.531670 (+ 383us) server_negotiation.cc:407] Connection header received
0811 00:32:34.531868 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:34.531874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:34.531982 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:34.532130 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:34.533168 (+ 1038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44234: BlockingRecv error: recv got EOF from 127.0.0.1:44234 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 00:32:34.667486 28833 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:32:34.665368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44236 (local address 127.2.120.61:44449)
0811 00:32:34.665661 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:34.665667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:34.665962 (+ 295us) server_negotiation.cc:407] Connection header received
0811 00:32:34.666306 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:34.666310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:34.666396 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:34.666508 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:32:34.667279 (+ 771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44236: BlockingRecv error: recv got EOF from 127.0.0.1:44236 (error 108)
Metrics: {"server-negotiator.queue_time_us":169}
I20260811 00:32:34.709936 28847 catalog_manager.cc:1259] Loaded cluster ID: 940fed5ae0db4c59be1bf16ae1d6ef98
I20260811 00:32:34.710057 28847 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1: loading cluster ID for follower catalog manager: success
I20260811 00:32:34.713057 28847 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1: acquiring CA information for follower catalog manager: success
I20260811 00:32:34.715075 28847 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e93a5682d58a468c88162d0029fd67c1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:32:34.746886 28863 catalog_manager.cc:1259] Loaded cluster ID: 940fed5ae0db4c59be1bf16ae1d6ef98
I20260811 00:32:34.747056 28863 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59: loading cluster ID for follower catalog manager: success
I20260811 00:32:34.749758 28863 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59: acquiring CA information for follower catalog manager: success
I20260811 00:32:34.752099 28863 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7689566d31074095b4a9b7f6fa004d59: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:32:34.934482 28833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:34.928284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44248 (local address 127.2.120.61:44449)
0811 00:32:34.928488 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:34.928493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:34.928664 (+ 171us) server_negotiation.cc:407] Connection header received
0811 00:32:34.928827 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:34.928831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:34.928997 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:34.929109 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:32:34.929562 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:34.930272 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:34.931915 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:34.932357 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:34.932470 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:34.933269 (+ 799us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:32:34.933866 (+ 597us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:32:34.933989 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:34.934173 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 00:32:34.934267 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:32:34.949429 28981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:34.941308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44304 (local address 127.2.120.60:40595)
0811 00:32:34.941945 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:34.941952 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:34.941974 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:32:34.942177 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:34.942183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:34.942402 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:34.942592 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:32:34.943494 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:34.944711 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:34.946235 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:34.946897 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:34.947033 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:34.947718 (+ 685us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:32:34.948591 (+ 873us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:32:34.948753 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:34.948940 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 00:32:34.949045 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":396,"threads_started":1}
I20260811 00:32:34.965531 28981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:34.958375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44308 (local address 127.2.120.60:40595)
0811 00:32:34.958631 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:34.958637 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:34.959237 (+ 600us) server_negotiation.cc:407] Connection header received
0811 00:32:34.959364 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:34.959370 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:34.959474 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:34.959565 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:32:34.960157 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:34.961292 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:34.963398 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:34.964049 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:34.964220 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:34.964751 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:32:34.964903 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:32:34.964968 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:34.965203 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 00:32:34.965303 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260811 00:32:34.965553 28833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:34.956570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44250 (local address 127.2.120.61:44449)
0811 00:32:34.956842 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:34.956852 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:34.957520 (+ 668us) server_negotiation.cc:407] Connection header received
0811 00:32:34.957857 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:34.957863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:34.957983 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:34.958080 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:32:34.958889 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:34.960036 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:34.962394 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:34.963154 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:34.963331 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:34.964166 (+ 835us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:32:34.964406 (+ 240us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:32:34.964632 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:34.964838 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 00:32:34.964974 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:32:34.968344 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:32:34.956692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56520 (local address 127.2.120.62:37597)
0811 00:32:34.957613 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0811 00:32:34.957619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:32:34.958233 (+ 614us) server_negotiation.cc:407] Connection header received
0811 00:32:34.958403 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:32:34.958412 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:32:34.958698 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:32:34.958899 (+ 201us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:32:34.959625 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:34.960501 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:34.964882 (+ 4381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:32:34.966014 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:32:34.966184 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:32:34.966856 (+ 672us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:32:34.967656 (+ 800us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:32:34.967796 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 00:32:34.967950 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 00:32:34.968048 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":656,"threads_started":1}
I20260811 00:32:34.974020 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 28586
I20260811 00:32:34.975733 28653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:32:35.091632 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 28586
I20260811 00:32:35.121553 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 28668
I20260811 00:32:35.123126 28735 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:32:35.236356 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 28668
I20260811 00:32:35.262144 2528 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskY6MNab/build/asan/bin/kudu pid 28749
I20260811 00:32:35.263806 28818 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:32:35.380173 2528 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY6MNab/build/asan/bin/kudu with pid 28749
2026-08-11T00:32:35Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5215 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10463 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (416078 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST